elasticsearchelasticsearch-ruby

Using percolate for bulk indexing in elasticsearch-ruby


According to this issue, elasticsearch supports using percolate with index (a single document) or bulk (multiple documents). No example is given for bulk, so I'm going by the issue's title that this functionality was added 5 years ago.

Unfortunately, I can't find any information about this functionality being available in the ruby API, elasticsearch-ruby.

Does anyone know if it's available, or perhaps have a code sample?

Thanks.

Update:

This page describes how to percolate while indexing a single document, and claims that it's possible in bulk. Now, how to do that in elasticsearch-ruby?

Looks like the NEST library can do it (see the bottom of the page), if I cared to rewrite my project in .Net.


Solution

  • There shouldn't be any special approach in Ruby, other than a simple _bulk operation specifying the _index as your index and as _type the .percolator:

    POST /_bulk
    {"index":{"_index":"some_index","_type":".percolator","_id":"1"}}
    {"query":{"match":{"whatever_field":"some value 1"}}}
    {"index":{"_index":"some_index","_type":".percolator","_id":"2"}}
    {"query":{"match":{"whatever_field":"some value 2"}}}
    {"index":{"_index":"some_index","_type":".percolator","_id":"3"}}
    {"query":{"match":{"whatever_field":"some value 3"}}}
    

    For sending multiple percolate requests, there is mpercolate which, initially, was created as a result of percolate in bulk feature request and ended up being called multi percolate api. And I see that elasticsearch-ruby has support for it.

    The ?percolate=* feature for bulk it seems is not in the list of arguments for _bulk and, in fact, there is an issue opened for this: github.com/elastic/elasticsearch-ruby/issues/176.

    LATER EDIT: I looked at this again and I'm more inclined to believe this feature has been removed completely in ES 1.0.0, following the percolator redesign github issue. I'm not seeing a specific statement about this being removed, but the source code related to bulk indexing with percolator option following that redesign has been deleted. Also, the documentation doesn't specify this option anywhere. Usually, when this happens is not an oversight but the functionality is simply not there. Still related to the documentation - don't forget that any resources you found online about this are at least two years old and I would take them with a grain of salt before assuming they are still valid.

    I wouldn't be surprised if you test this in Elasticsearch (no ruby, nest or whatever client) with curl or Sense and see that it doesn't work. I tested this in 1.6 just now and I'm not seeing the functionality.