Netsuite Saved Searches API not returning full set of rows

I've been assigned the task of extracting data from saved searches to a Virtual Machine (VM). During my research, I discovered Tim Dietrich's method for accomplishing this. I implemented his approach, which involves using a RESTlet and calling it through a Python script. Initially, the process seemed successful.

However, upon comparing the row counts between the API extract and the standard Excel export from NetSuite, I noticed a significant discrepancy. The API extract yielded a considerably lower number of rows compared to the Excel workbook.

This unexpected difference in data volume has raised concerns about the completeness of the API extraction method. I'm now seeking to understand the cause of this discrepancy and find a solution to ensure accurate and complete data extraction from NetSuite's saved searches.

Upon further investigation, I discovered that the issue might be related to pagination. My research indicated that pagination needs to be implemented within the RESTlet itself. Acting on this information, I modified the JavaScript code for the RESTlet and redeployed it in NetSuite. However, despite these changes, the row count from the API extract remains lower than expected.

var 
    log,
    query,
    response = new Object();    


define( [ 'N/log', 'N/query' ], main );


function main( logModule, queryModule ) {

    log = logModule;
    query = queryModule;

    return { post: postProcess }

}


function postProcess( request ) {   
    
    try {
    
        if ( ( typeof request.query == 'undefined' ) || ( request.query === null ) || ( request.query == '' ) ) {       
            throw { 'type': 'error.SuiteAPIError', 'name': 'INVALID_REQUEST', 'message': 'No query was specified.' }
        }   
    
        if ( typeof request.params == 'undefined' ) { request.params = new Array(); }
                    
        response.rows = query.runSuiteQL( { query: request.query, params: request.params } ).asMappedResults(); 
                            
        return response;
                
    } catch( e ) {  
        log.debug( { 'title': 'error', 'details': e } );
        return { 'error': { 'type': e.type, 'name': e.name, 'message': e.message } }
    }   
        
}

Your posted code doesn't show any use of the paging API.

In case it helps a simple example of that is:

function processQuery(queryStr){
    const results = query.runSuiteQLPaged({
        query: queryStr,
        pageSize: 1000
    });

    let data = [];
    results.iterator().each((res)=>{
        data = data.concat(res.value.data.asMappedResults());
        return true;
    });

    return data;
};