sortingelasticsearchcase-insensitive

Unexpected (case-insensitive) string sorting in Elasticsearch


I have a list of console platforms that I'm sorting in Elasticsearch.

Here is the mapping for the "name" field:

{
    "name": {
        "type": "multi_field",
        "fields": {
            "name": {
                "type": "string",
                "index": "analyzed"
            },
            "sort_name": {
                "type": "string",
                "index": "not_analyzed"
            }
        }
    }
}

When I execute the following query

{
  "query": {
    "match_all": {}
  },
    "sort": [
        {
          "name.sort_name": { "order": "asc" }
        }
    ],
    "fields": ["name"]
}

I get these results:

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "failed": 0
    },
    "hits": {
        "total": 17,
        "max_score": null,
        "hits": [
            {
                "_index": "platforms",
                "_type": "platform",
                "_id": "1393602489",
                "_score": null,
                "fields": {
                    "name": "GameCube"
                },
                "sort": [
                    "GameCube"
                ]
            },
            {
                "_index": "platforms",
                "_type": "platform",
                "_id": "1393602490",
                "_score": null,
                "fields": {
                    "name": "Gameboy Advance"
                },
                "sort": [
                    "Gameboy Advance"
                ]
            },


    {
            "_index": "platforms",
            "_type": "platform",
            "_id": "1393602498",
            "_score": null,
            "fields": {
                "name": "Nintendo 3DS"
            },
            "sort": [
                "Nintendo 3DS"
            ]
        },

        ...remove for brevity ...

        {
            "_index": "platforms",
            "_type": "platform",
            "_id": "1393602493",
            "_score": null,
            "fields": {
                "name": "Xbox 360"
            },
            "sort": [
                "Xbox 360"
            ]
        },
        {
            "_index": "platforms",
            "_type": "platform",
            "_id": "1393602502",
            "_score": null,
            "fields": {
                "name": "Xbox One"
            },
            "sort": [
                "Xbox One"
            ]
        },
        {
            "_index": "platforms",
            "_type": "platform",
            "_id": "1393602497",
            "_score": null,
            "fields": {
                "name": "iPhone/iPod"
            },
            "sort": [
                "iPhone/iPod"
            ]
        }
    ]
}

Everything is sorted as expected except the iPhone/iPod result is at the end (instead of after GameBoy Advance) - why does the / in the name have an effect on the sorting?

Thanks


Solution

  • Okay so I discovered the reason wasn't anything to do with the /. ES will sort by capital letters then lower case letters.

    I added a custom analyzer to the settings of the index creation:

    {
        "analysis": {
            "analyzer": {
                "sortable": {
                    "tokenizer": "keyword",
                    "filter": [
                        "lowercase"
                    ]
                }
            }
        }
    }
    

    Then in the field mapping I added 'analyzer': 'sortable' to the sort_name multi field.