objective-cxcodecore-datansxmlparsermagicalrecord

Objective-c NSXMLParser with Coredata takes too long to parse and store data


I'm not a experience xcode/objective-c programmer so I'm sorry if you might not understand somethings I might say, or somethings I might say wrong.

So everything started with apple rejecting our app because it says:

"Your app did not include sufficient content in the binary for the app to function at launch, and we were required to download or unpack additional resources before we could use it."

Because our app is a transports app, it needs to download dynamic data from services to keep the app with the most accurate and updated data. So basically everytime we open the app we ask the user to download the data (~2.5 MB), but since Apple refused to aprove the app I made an exception and let user enter without downloading any data, but it needs to convert the local XML file into core-data database.

My problem is, this file that is like 2,5 MB space with ~17k lines takes at least 2 minutes to read and store the provided data.

So I tried to see the parser if it was the problem but the code seems fine to me.

I know that this what I'm doing might not be the solution because of what Apple said the "unpacking additional resources" so I think it won't pass the app verification, but still I wanted this to do the parse and store with less time...

This is my code:

- (void)parser:(NSXMLParser *)parser didStartElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qualifiedName attributes:(NSDictionary *)attributeDict {
    //    <Network LinesNumber="28" ZonesNumber="112" StopsNumber="114">
    if ([elementName isEqualToString:@"Network"]) {
        int ln = [[attributeDict objectForKey:@"LinesNumber"] intValue];
        int zn = [[attributeDict objectForKey:@"ZonesNumber"] intValue];
        int sn = [[attributeDict objectForKey:@"StopsNumber"] intValue];
        totalItems = (totalItems + ln + sn + zn) *1.0;
        updateaction(0);
    }
    else if ([elementName isEqualToString:@"Stop"]) {
        int idStop = [[attributeDict objectForKey:@"Id"] intValue];
        NSString* name = [[attributeDict objectForKey:@"Name"] capitalizedString];
        NSString* codName = [attributeDict objectForKey:@"CodName"];
        int idZona = [[attributeDict objectForKey:@"IdZona"] intValue];
        int idDistrito = [[attributeDict objectForKey:@"IdCounty"] intValue];
        int idConcelho = [[attributeDict objectForKey:@"IdDistrict"] intValue];
        int idFreguesia = [[attributeDict objectForKey:@"IdParish"] intValue];
        double latitude = [[attributeDict objectForKey:@"CoordYY"] doubleValue];
        double longitude = [[attributeDict objectForKey:@"CoordXX"] doubleValue];
        NSString* obs = [attributeDict objectForKey:@"Observation"];

        OperatorZone *zone = [Database operatorZoneFromId:idZona];

        Stop *stop = [Database createStop:idStop withName:name withCodName:codName withIdZona:idZona withIdDistrito:idDistrito withIdConcelho:idConcelho withIdFreguesia:idFreguesia withLatitude:latitude withLongitude:longitude withObservations:obs withOperatorZone:zone];
        [stop setCicID:cicID];
        [stop setOperatorID:operatorID];
        NSLog(@"Saving stop with Name: %@, cicID: %@, operatorID: %@", name, cicID, operatorID);

        [stops_dict setObject:stop forKey: [NSNumber numberWithInt:idStop]];

        if (zone != nil) {
            [zone addStopsObject:stop];
          //  [[zone managedObjectContext] MR_saveToPersistentStoreAndWait]; // TIRAR ISTO DAQUI E POR NO FIM DE TUDO
        }

        itemcount++;
        progress = itemcount/totalItems;
        updateaction(progress);
    }

    else if ([elementName isEqualToString:@"Line"]) {
//        NSLog(@"Checking line..");
        int sid = [[attributeDict objectForKey:@"Id"] intValue];
        NSString * name = [attributeDict objectForKey:@"LineName"];
        NSString * returnName = [attributeDict objectForKey:@"ReturnLineName"];
        NSString * companyID = [attributeDict objectForKey:@"CompanyId"];
        int isCircular = [[attributeDict objectForKey:@"IsCircular"] boolValue];
        int idOperator = [[attributeDict objectForKey:@"IdOperator"] intValue];
        NSString * version = [attributeDict objectForKey:@"Version"];
        currentLine = [Database createLine:sid withName:name withReturnName:returnName isCircular:isCircular withOperatorID:idOperator withCompanyID:companyID withVersion:version];
        latestLineOpID = idOperator;
        [currentLine setCicID:cicID];
        [currentLine setOperatorID:operatorID];
        lineWithOwnStops = (idOperator == suboperatorid);
        itemcount++;
        progress = itemcount/totalItems;
        updateaction(progress);
    }

The XML File data is like this:

<Network CountiesNumber="0" ContactsNumber="2" LinesNumber="326" StopsNumber="3161" ZonesNumber="2866">
    <Zones>
        <OperatorZone Name="Cavadas (R 25 Abril, 60) Café O Renascer" Id="20274" />
    </Zones>
    <Stops>
        <Stop Id="108591" Name="Setúbal (Avª 22 Dezembro, 25)" CodName="2" IdZona="22793" CoordXX="-8.89310700" CoordYY="38.52755000" Observation="" />
    </Stops>
    <Lines>
        <Line ReturnLineName="Cacilhas - Cristo Rei" LineName="Cristo Rei - Cacilhas" IsCircular="false" CompanyId="101" IdOperator="84" Id="16344" Version="05-08-2019 00:00:00">
            <StopLines>
                <StopLine StopName="0" OrderPath_I="1" OrderPath_V="0" ZoneId="20435" Id="56356194" IdStop="109346" />
                <StopLine StopName="0" OrderPath_I="2" OrderPath_V="0" ZoneId="20423" Id="56356195" IdStop="109838" />
            </StopLines>
        </Line>
    </Lines>
</Network>

EDIT - Example Database Method:

+ (Stop *)createStop:(int)id withName:(NSString*)name withCodName:(NSString *)codName
withIdZona:(int)idZona
withIdDistrito:(int)idDistrito
withIdConcelho:(int)idConcelho
withIdFreguesia:(int)idFreguesia
withLatitude:(double)latitude
withLongitude:(double)longitude
withObservations:(NSString *)observations
withOperatorZone:(OperatorZone *)operator
{
    Stop * stop = [Stop MR_createEntity];
    stop.ownStop = false;
    stop.name = name;
    stop.codName = codName;
    stop.idZona = [NSNumber numberWithInt:idZona];
    stop.idDistrito = [NSNumber numberWithInt:idDistrito];
    stop.idConcelho = [NSNumber numberWithInt:idConcelho];
    stop.idFreguesia = [NSNumber numberWithInt:idFreguesia];
    stop.id = [NSNumber numberWithInt:id];
    stop.latitude = [NSNumber numberWithDouble:latitude];
    stop.longitude = [NSNumber numberWithDouble:longitude];

    if (idDistrito != 0){
        NSLog(@"bla bla bla");
    }
    stop.distrito = [Distrito MR_findFirstByAttribute:@"id" withValue:[NSNumber numberWithInt:idDistrito]];
    stop.concelho = [Concelho MR_findFirstByAttribute:@"id" withValue:[NSNumber numberWithInt:idConcelho]];
    stop.freguesia = [Freguesia MR_findFirstByAttribute:@"id" withValue:[NSNumber numberWithInt:idFreguesia]];

    stop.operatorzone = operator;
    //ac    [[stop managedObjectContext] MR_saveToPersistentStoreAndWait];
    //    NSLog(@"Stop %d - %@ saved", [stop.id intValue], stop.name);
    return stop;
}

Theres also an example of a parser that this most likely looks as mine: https://gist.github.com/xslim/1020767

Difference is, that hes using NSEntityDescription insertNewObjectForEntityForName: and I'm using MR_createEntity


Solution

  • I finally found the solution. So I decided to go with a pre-loaded DB instead of filling the data of my database from a local xml file.

    Basically what I done was:

    And then in my AppDelegate file after didFinishLaunchingWithOptions: I added this code:

    // This code is needed to preload the database
    
    NSArray *paths = [[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask];
    NSURL *documentPath = [paths lastObject];
    NSURL *storeURL = [documentPath URLByAppendingPathComponent:@"MyDB.sqlite"];
    
    // Check if the database already exists in the document folder, if it doesn't exist add it to the documents folder
    if (![[NSFileManager defaultManager] fileExistsAtPath:[storeURL path]]) {
        NSURL *preloadURL = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:@"MyDB" ofType:@"sqlite"]];
        NSError* err = nil;
    
        if (![[NSFileManager defaultManager] copyItemAtURL:preloadURL toURL:storeURL error:&err]) {
            NSLog(@"Error: Unable to copy preloaded database.");
        }
    }
    
    // Setup Magical Record in the document path
    [MagicalRecord setupCoreDataStackWithAutoMigratingSqliteStoreAtURL:storeURL];
    

    For some reason the setupCoreDataStackWithAutoMigratingSqliteStoreNamed wasen't working for me (Maybe because was getting the MyDB.sqlite files from the other targets? I don't know), so I decided to go with setupCoreDataStackWithAutoMigratingSqliteStoreAtURL