javascriptnode.jsjobsparse-serverparse-javascript-sdk

Parse-Server XMLHttpRequest failed on save with large dataset


I'm running Parse-Server v2.2.12 and am trying to run a job that pulls a huge dataset (~13k records), processes it and then saves each record to a table in my Parse db. It appears that my server is unable to accept all the save requests. I get a code 100 error with the message: XMLHttpRequest failed: "Unable to connect to the Parse API" for about 60% of the save requests.

My code looks something like this:

makeHttpRequest('GET', endpoint).then( (httpResponse) => {

    const dataArray = httpResponse.body.array;
    const Class = Parse.Object.extend('className');

    const savePromises = _.map(dataArray, (jsonObject) => {

        const object = new Class();
        object.set(IdKey, "" + jsonObject.id);
        object.set(nameKey, jsonObject.name);
        return object.save(null, {useMasterKey: true});
    });

    return Parse.Promise.when(savePromises);
});

I've tried breaking the array up into thirds (~4.5k records) and performing each set of saves sequentially using promise.then but I'm still getting:

{
  "code": 100,
  "message": "XMLHttpRequest failed: \"Unable to connect to the Parse API\""
}

Update: After migrating this job to our production server which is https-enabled, I get a slightly different error message and many more of the saves complete. I suspect more saves complete due to the additional capacity of our production servers. The new error looks like:

{
  "code": 100,
  "message": "XMLHttpRequest failed: {\"UNSENT\":0,\"OPENED\":1,\"HEADERS_RECEIVED\":2,\"LOADING\":3,\"DONE\":4,\"readyState\":4,\"responseText\":\"\",\"responseXML\":\"\",\"status\":504,\"statusText\":null,\"withCredentials\":false}"
}

Solution

  • I solved this by using Parse.Object.saveAll(objects). Now I don't have any problem saving all 13k records in one fell swoop. Code looks like this:

    makeHttpRequest('GET', endpoint).then( (httpResponse) => {
    
        const dataArray = httpResponse.body.array;
        const Class = Parse.Object.extend('className');
    
        const allObjects = _.map(dataArray, (jsonObject) => {
    
            const object = new Class();
            object.set(IdKey, "" + jsonObject.id);
            object.set(nameKey, jsonObject.name);
            return object;
        });
    
        return Parse.Object.saveAll(allObjects, {useMasterKey: true});
    });
    

    Everything seems to be working fine now.