javascripthtmlvue.jspos

js search in array with 50k result is so slow

i have web app pos the array filter search in client side in browser not in the server first i download all products as array and work on it and its working so good with number of products under 20k since a while the array of the products get the number of 50k

and my results array like this ..

   [
      {
        "id": 1,
        "name": "Product 1",
        "number": "1",
        "available_on_pos": 1,
        "pos_quick_lunch": 1,
        "product_category_id": 1,
        "discount_value": 10,
        "smallest_unit_id": 1,
        "smallest_unit_barcode": null,
        "smallest_unit_selling_price": 10,
        "smallest_unit_selling_price_above": null,
        "smallest_unit_selling_price_above_price": null,
        "tax_value": 16,
        "get_smallest_unit_id": {
          "id": 1,
          "name": "Unit"
        },
        "get_price_list_lines": []
      },
      {
        "id": 13,
        "name": "Product 2",
        "number": "13",
        "available_on_pos": 1,
        "pos_quick_lunch": null,
        "product_category_id": 1,
        "discount_value": null,
        "smallest_unit_id": 1,
        "smallest_unit_barcode": "9501025172815",
        "smallest_unit_selling_price": 0.509,
        "smallest_unit_selling_price_above": null,
        "smallest_unit_selling_price_above_price": null,
        "tax_value": 16,
        "get_smallest_unit_id": {
          "id": 1,
          "name": "Unit"
        },
        "get_price_list_lines": []
      }
    ]

and this is the search method

var results = this.products.filter(obj => obj.smallest_unit_barcode != null && obj.smallest_unit_barcode.includes(value));
this.pos_quick_lunch = results;

as i say the the products array is 50k the search like have delay when i press barcode gun multiable time at same product search and this is long time in pos to wait .. is there any way thats i can make the search much easer and faster

Solution

  • is there any way thats i can make the search much easer and faster

    var results = this.products.filter(obj => obj.smallest_unit_barcode != null && obj.smallest_unit_barcode.includes(value));

    As long as we keep the focus on this part of code and don't start discussions about general design ideas, I see a these options...

    Cache the undefined case

    Your could create a list of products with smalles_unit_barcode != null or undefined and keep it. This should give you a small improvement.

    // do this only when products collection changes
const this.cachedProductsWithBarcode = this.products.filter(p => !!p.smallest_unit_barcode);

// do this on every request / input
const results = this.cachedProductsWithBarcode(p => p.smallest_unit_barcode.includes(value));

    Do you need partial matches?

    If you get the full article number / gtin, there's no need to do a partial search and you could use a dictionary lookup instead.

    // only when products collection changes
const dict = this.products.reduce((prev, cur) => (prev[cur.smalles_unit_barcode] = cur, prev), {});

// for each request
return dict[value];

    (If you get the full barcode most of the time but some times just a part of it, you could think about doing a lookup first and continue with a search if you have no match.)

    Delay user input / Cache

    If you search needle comes from a user input, where users can type the barcode and you do this search on every keystroke this search will block the UI making the app unusable. Delaying the user input (= do the search only if we get no more key strokes for e.g. 500ms) could improve the user experience.

    In addition you could cache results

    // User input "123"
// const firstResult = products.find(...)

// User input "1234"
// ... must be in firstResult, so let' search there
return firstResult.find(...)

    (This makes only sense if it's 'search while typing' and you need the invalidate the cache if needle allows more results than cached results list.)

    Misc...