arraysjsonjsonobjectrequestjson-arrayagg

What is the best way of getting index from huge JSON Array of objects that matches with array of JSON objects


I want to know the best way of getting indexes from huge (array length is nearly 150 000) JSON Array of JSON objects that matches with array of JSON objects.

Currently I'm using for loop to do this job but it is taking 3 to 4 minutes of time. Is there any best way that I can improve the performance? Below is an example that illustrate my requirement.

var hugeArray = [
    {firstName:"John", lastName:"Doe", age:21},
    {firstName:"Abraham", lastName:"Lincoln", age:46},
    {firstName:"Andy", lastName:"Crossland", age:32},
    .
    .
    .
    {firstName:"Jimmy", lastName:"Fletcher", age:65}
];

Assume length of an hugeArray is 150 000. And I've another JSON array(matchArray) of length 15 000.

var matchArray = [
    {firstName:"John", lastName:"Doe"},
    {firstName:"Andy", lastName:"Crossland"},
    .
    .
    .
    {firstName:"Jimmy", lastName:"Fletcher"}
];

I want indexes from hugeArray that matches with matchArray.

For example from the above data, matchArray has list of objects and these objects to be matched with the objects in the hugeArray. If matched then return indexes of the hugeArray. Output of above example will be as below.

[0,2,...,150000]

For this I've written a function and that function returns index of hugeArray. If matchArray object matches with hugeArray then return the index of matched hugeArray otherwise returns -1.

//returns -1 if no matched objects found otherwise returns index of searchArray that matched with obj.
//searchArray : Array that used to search in and return matched index.
//obj : JSON object which is used to match in with searchArray objects.
function getArrayIndex(searchArray,obj){
    var index = -1;
    for(var i=0,searchArrayLen=searchArray.length;i<searchArrayLen;i++){
        var keyMatch = true;
        for(var key in obj){
            if(obj[key].trim() !== searchArray[i][key].trim()){
                keyMatch = false;
                break;
            }
        }
        if(keyMatch){
            index = i;
            return index;
        }
    }
    return index;
}

Wrote below getIndexes function to call above getArrayIndex function.

//returns array of indexes.
function getIndexes(hugeArray,matchArray){
    var indexArray = [];
    var matchArrayLen = matchArray.length;
    for(var i=0; i<matchArrayLen; i++){
        var matchIndex = getArrayIndex(hugeArray,matchArray[i]);
        if(matchIndex !== -1){
            indexArray.push(matchIndex);
        }
    }
    return indexArray;
}

Finally by calling getIndexes function, giving the all matched indexes in array format.

For example,

var index = getIndexes(hugeArray,matchArray);
console.log(index);// This prints an array that contains matched indexes.
//For above data, output will be like [0,2,...,150000].

Solution

  • You can use Map Object for this.

    let theMap = new Map(hugeArray.map((item, index) => [item.firstName + " " + item.lastName, index]));
    
    let result = matchArray.map(item => theMap.get(item.firstName + " " + item.lastName))
      .filter(i => i !== undefined)
    

    This assumes firstName or lastName do not have spaces. Use another char if they might include spaces.

    If hugeArray has duplicates:

    let theMap = new Map();
    hugeArray.forEach((item, index) => {
      let key = item.firstName + " " + item.lastName;
      let value = theMap.get(key);
      if (value === undefined)
        theMap.set(key, [index]);
      else
        value.push(index);
    });
    
    let result = matchArray.flatMap(item => theMap.get(item.firstName + " " + item.lastName))
      .filter(i => i !== undefined);
    

    If we need a sorted result when hugeArray has duplicates:

    let result = matchArray.flatMap(item => theMap.get(item.firstName + " " + item.lastName))
          .filter(i => i !== undefined).sort();