amazon-web-serviceselasticsearchkibanaaws-elasticsearchopensearch

Why is Elasticsearch/Opensearch query returning everything?


I'm using AWS's OpenSearch, and I'm having trouble getting any queries or filters to only return matching results.

To test, I'm using sample ecommerce data that includes the field "customer_gender" that's one of "MALE" or FEMALE." I'm trying to use the following query:

GET /kibana_sample_data_ecommerce/_search

{
  "query": {
    "multi_match": {
      "query" : "FEMALE"
      "fields": ["customer_gender"]
    }
  }
}

But all the results with "MALE" are still showing up. I've tried this as a POST request instead of a GET request, I've tried lowercasing, and I've tried using term queries and fulltext queries (like match) and I'm getting the same results every time. Is there some way I'm formatting these queries wrong? I've scoured through the documentation and I haven't been able to find any clues that have had a difference.

Here are some JSON examples of other types of queries I've tried:

{
  "query": {
    "match": {
      "customer_gender": "FEMALE"
    }
  }
}

{
  "query": {
    "match": {
      "customer_gender": {
        "value": "FEMALE"
      }
    }
  }
}

{
  "query": {
    "constant_score": {
      "filter": {
        "term": {
          "customer_gender": "FEMALE"
        }
      }
    }
  }
}

Here is an example of a _source for query objects:

        {
          "category" : [
            "Men's Clothing"
          ],
          "currency" : "EUR",
          "customer_first_name" : "Eddie",
          "customer_full_name" : "Eddie Underwood",
          "customer_gender" : "MALE",
          "customer_id" : 38,
          "customer_last_name" : "Underwood",
          "customer_phone" : "",
          "day_of_week" : "Monday",
          "day_of_week_i" : 0,
          "email" : "eddie@underwood-family.zzz",
          "manufacturer" : [
            "Elitelligence",
            "Oceanavigations"
          ],
          "order_date" : "2021-11-29T09:28:48+00:00",
          "order_id" : 584677,
          "products" : [
            {
              "base_price" : 11.99,
              "discount_percentage" : 0,
              "quantity" : 1,
              "manufacturer" : "Elitelligence",
              "tax_amount" : 0,
              "product_id" : 6283,
              "category" : "Men's Clothing",
              "sku" : "ZO0549605496",
              "taxless_price" : 11.99,
              "unit_discount_amount" : 0,
              "min_price" : 6.35,
              "_id" : "sold_product_584677_6283",
              "discount_amount" : 0,
              "created_on" : "2016-12-26T09:28:48+00:00",
              "product_name" : "Basic T-shirt - dark blue/white",
              "price" : 11.99,
              "taxful_price" : 11.99,
              "base_unit_price" : 11.99
            },
            {
              "base_price" : 24.99,
              "discount_percentage" : 0,
              "quantity" : 1,
              "manufacturer" : "Oceanavigations",
              "tax_amount" : 0,
              "product_id" : 19400,
              "category" : "Men's Clothing",
              "sku" : "ZO0299602996",
              "taxless_price" : 24.99,
              "unit_discount_amount" : 0,
              "min_price" : 11.75,
              "_id" : "sold_product_584677_19400",
              "discount_amount" : 0,
              "created_on" : "2016-12-26T09:28:48+00:00",
              "product_name" : "Sweatshirt - grey multicolor",
              "price" : 24.99,
              "taxful_price" : 24.99,
              "base_unit_price" : 24.99
            }
          ],
          "sku" : [
            "ZO0549605496",
            "ZO0299602996"
          ],
          "taxful_total_price" : 36.98,
          "taxless_total_price" : 36.98,
          "total_quantity" : 2,
          "total_unique_products" : 2,
          "type" : "order",
          "user" : "eddie",
          "geoip" : {
            "country_iso_code" : "EG",
            "location" : {
              "lon" : 31.3,
              "lat" : 30.1
            },
            "region_name" : "Cairo Governorate",
            "continent_name" : "Africa",
            "city_name" : "Cairo"
          },
          "event" : {
            "dataset" : "sample_ecommerce"
          }
        }

EDIT: The Kibana dashboards/explore pages are all querying just fine, as a note, but when I copy-paste the same queries to the console, it's still not working there


Solution

  • The problem is that you have an empty line between GET and the query, so there's no query being sent, hence it's equivalent to a match_all query:

    GET /kibana_sample_data_ecommerce/_search
                             <--- remove this empty line
    {
      "query": {
        "multi_match": {
          "query" : "FEMALE"
          "fields": ["customer_gender"]
        }
      }
    }