elasticsearchelasticsearch-dslelasticsearch-date

Partial search on date fields in elasticsearch


I'm trying to implement partial search on a date field in elastic search. For example if startDate is stored as "2019-08-28" i should be able to retrieve the same while querying just "2019" or "2019-08" or "2019-0".

For other fields i'm doing this:

{
            "simple_query_string": {
              "fields": [
                "customer"
              ], 
              "query": "* Andrew *",
              "analyze_wildcard": "true",
              "default_operator": "AND"
            }}

which perfectly works on text fields, but the same doesn't work on date fields.

This is the mapping : {"mappings":{"properties":{"startDate":{"type":"date"}}}}

Any way this can be achieved, be it change in mapping or other query method? Also i found this discussion related to partial dates in elastic, not sure if it's much relevant but here it is:

https://github.com/elastic/elasticsearch/issues/45284


Solution

  • Excerpt from ES-Docs

    Internally, dates are converted to UTC (if the time-zone is specified) and stored as a long number representing milliseconds-since-the-epoch.

    It is not possible to do searching as we can do on a text field. However, we can tell ES to index date field as both date & text.e.g

    Index date field as multi-type:

    PUT sample
    {
      "mappings": {
        "properties": {
          "my_date": {
            "type": "date",
            "format": "year_month_day",//<======= yyyy-MM-dd
            "fields": {
              "formatted": {
                "type": "text", //<========= another representation of type TEXT, can be accessed using my_date.formatted
                "analyzer": "whitespace" //<======= whitespace analyzer (standard will tokenized 2020-01-01 into 2020,01 & 01)
              }
            }
          }
        }
      }
    }
    
    POST dates/_doc
    {
      "date":"2020-01-01"
    }
    
    POST dates/_doc
    {
      "date":"2019-01-01"
    }
    

    Use wildcard query to search: You can even use n-grams at indexing time for faster search if required.

    GET dates/_search
    {
      "query": {
        "wildcard": {
          "date.formatted": {
            "value": "2020-0*"
          }
        }
      }
    }