c++xmlqtqdomdocument

Error while loading larger files with setContent(&file) at a qt DOM xml Parser


I'm running into a strange problem which i cannot solve. It seems nobody has this problem. While loading a small XML file (4mb) everything ist fine and the programm runs normal, but when i'm trying to load a larger file (200mb) the programm crashes without any error (even in debug mode). It also does not print out the error messages since the programm crashes before their call. Thanks for helping.

Errorlog from the QT creator is:

The program has unexpectedly finished. C:/Parser [path]/XmlDOM crashed

Code:

QFile file("./file.osm");
qDebug() << file.exists();
qDebug() << file.size();

QString errorStr;
int errorLine;
int errorColumn;

QDomDocument document;

if(!file.open(QIODevice::ReadOnly | QIODevice::Text))
{
    qDebug() << "Failed to open file";
    return -1;
}
else
{
    if(!document.setContent(&file, false, &errorStr, &errorLine, &errorColumn)) //here the programm crashes
    {
        std::cerr << "Error: Parse error at line " << errorLine << ", "
                          << "column " << errorColumn << ": "
                          << qPrintable(errorStr) << std::endl;
       return -1;
    }
    qDebug() << file.isReadable(); //with small files this becomes true 
    file.close();
    } 
QDomElement root = document.firstChildElement();

Solution:

Activate a swap partition or get more RAM, the program chrashes because the PC is/was out of RAM. The updates are left in the question only for the reason to document my further steps.

Update: I installed everything on another machine. Now im getting some output:

Error: Parse error at line 1, column 1: unexpected end of file

Funny thing now even the small files are not working and return the Error. After some digging some people set the path to the file wrong so i checked my path.

qDebug() << "File exists: " << file.exists(); 
qDebug() << "File path: " << QFileInfo(file).absoluteFilePath();
qDebug() << "File size: " << file.size();    

This returns: true, /path/to/file/file.osm , correct size

I also checked if my XML files are valid and they are. So any new suggestions? So far im stuck

Update2: First thank you for your answers! One desperate attempt was:

else{
  document.setContent(&file); //passed and worked, funny
}
qDebug() << file.isReadable(); 
file.close();

This finally works with the larger and smaller files on the new setup:

else
{

    if(!document.setContent(&file))
    {
       std::cerr << "Error: Open file "<< std::endl;
       return -1;
    }
    qDebug() << file.isReadable(); 
    file.close();

Why i used a DOM Parser: The XML Structure is like this:

<osm>
    <node id ="1" lat="value", lon="value" />
    <node id ="2" lat="value", lon="value" />
    <node id ="3" lat="value", lon="value" />

    <way id="12345">
      <nd ref ="1"/>
      <nd ref ="2"/>
      <nd ref ="3"/>
    </way>
</osm>

I want to rebuild the way points for that the lat,lon values from the node´s are necessary. For this i want to be able to match the way ref id with the node id and get the values into the way. Is the Sax parser the better solution for this? I tought with the DOM tree i could easy go through the "node"s and match the id´s without parsing the complete XML again. I´m using Ubuntu and with the new setup qt5. I got a i5 2nd generation and 8GB of RAM which are full when the big file is being processed. One run in release mode needs 50 minutes für 1000ways with 5-50 nodes.


Solution

  • The solution is activate a swap partition or get more RAM, the program chrashed because the PC is/was out of RAM. A even better solution is to use a SAX parser.

    To the runtime Problem: use maps instead of vectors. The runtime scaled down to ~20 secs for a big file.