elasticsearchopensearch

LTR logs missing when using the rescore block in OpenSearch


I'm facing an issue where using an SLTR query in the "rescore" part of my request, results in no logs being returned in the response! However, if I put the same SLTR query in the "query" part of the request, everything works perfectly and I receive the expected logs. So as an example:

{
  "query": {
    "bool": {
      "must: [
        ..., // bunch of other queries,
        { "sltr": {
            "_name": "<query_name>",
            "params": {},
            "featureset": "my_featureset"
          }
        }
      ]
    }
  },
  "size": 30,
  "ext": {
    "ltr_log": {
      "log_specs": {
        "name": "<log-id>",
        "named_query": "<query-name>"
      }
    }
  },
  ... // other specifications
}

Does return the logs, but this one doesn't:

{
  "query": {
    "bool": {
      "must: [
        ..., // bunch of queries
      ]
    }
  },
  "size": 30,
  "rescore": {
    "window_size": 10,
    "query": {
      "rescore_query": {
        { "sltr": {
            "_name": "<query_name>",
            "params": {},
            "featureset": "my_featureset"
          }
        }
      }
    }
  },
  "ext": {
    "ltr_log": {
      "log_specs": {
        "name": "<log-id>",
        "rescore_index": 0
      }
    }
  },
  ... // other specifications
}

but of course, the latter one is much more performant, and that's what I really want to get at the end, only rescoring the top 10 documents. And the exact same query works perfectly fine on Elasticsearch.

Has anyone ever encountered this? I appreciate any help.


Update: If I change the "must" in my query to "filter" (in my first example), and put the sltr query inside query block, then I get the logs back, and it doesn't really affect the performance! But I think still what I want at the end is the sltr query only getting applied to the documents when rescoring.


Solution

  • Well, it's been quite some time now since I posted this question, and we came up with this solution which works for us. IDK why, for some reason opensearch (or the plugin that we're using) doesn't pick up the logs from the rescore phase, so we added the sltr query in "query.filter" as well and gave it a name, so that opensearch can pick up the logs from there, and it doesn't affect the scores (since it's inside filter). So here's the structure of our request now:

    {
      "query": {
        "bool": {
          "must: [...],
          "filter": [
            {
              "sltr": {
                "_name": "<query_name>",
                "params": {},
                "featureset": "my_featureset"
              }
            }
          ]
        }
      },
      "rescore": {
        "window_size": 10,
        "query": {
          "rescore_query": {
            { "sltr": {
                "params": {},
                "featureset": "my_featureset"
              }
            }
          }
        }
      },
      "ext": {
        "ltr_log": {
          "log_specs": {
            "name": "<log-id>",
            "named_query": "<query-name>"
          }
        }
      }
    }
    

    I hope it helps someone :)