elasticsearchelasticsearch-dsl

Elastic search query not returning expected responses using dsl in elastic 8.8.2


I have two queries two select from an index document. the "applications" is a nested path within "candidate". I search amongst the field workexp, summary , but the documents contains the value in the Felds, it does not show up in the below query 1. It always shows up in response of query 2. query 1:

 {
    "nested": {
        "path": "applications",
        "query": {
            "bool": {
                "must": [
                    {
                        "simple_query_string": {
                            "default_operator": "and",
                            "fields": [
                                "applications.workexp"
                            ],
                            "query": "(Tableau) Or (Tableau) Or (Tableau workbooks)"
                        }
                    },
                    {
                        "simple_query_string": {
                            "default_operator": "and",
                            "fields": [
                                "applications.summary"
                            ],
                            "query": "(Tableau) Or (Tableau) Or (Tableau workbooks)"
                        }
                    },
                    {
                        "match": {
                            "applications.forreqid": {
                                "query": "FH-REQ-3"
                            }
                        }
                    }
                ]
            }
        }
    }
}

query 2

{
    "nested": {
        "path": "applications",
        "query": {
            "bool": {
                "must": [
                    {
                        "match": {
                            "applications.forreqid": {
                                "query": "FH-REQ-3"
                            }
                        }
                    }
                ],
                "must_not": [
                    {
                        "simple_query_string": {
                            "default_operator": "and",
                            "fields": [
                                "applications.workexp"
                            ],
                            "query": "(Tableau) Or (Tableau) Or (Tableau workbooks)"
                        }
                    },
                    {
                        "simple_query_string": {
                            "default_operator": "and",
                            "fields": [
                                "applications.summary"
                            ],
                            "query": "(Tableau) Or (Tableau) Or (Tableau workbooks)"
                        }
                    }
                ]
            }
        }
    }
}

The document I am expecting to come in the response

{
    createdby : abc@soething.com, 
    applications : [
        {
        applnid : yy, 
        summary : top and Tableau Server.· Involved in dashboard test cases creation and execution, prepared the understanding and function/ process flow documents on various dashboards for the end users.· Created technical specifications document as well as functional documents in support of the user requirements.· Generate Tableau reports to analyze data from multiple data sources like Oracle, SQL Server, Excel, Flat Files, etc · Experienced in designing customized interactive dashboards in Tableau using Marks, Action, filters, parameter and calculations · Having experience in Tableau Desktop Creating Sets, Group, Sort, Parameter, Quick filters, Context Filters, Data blending, Joins and Calculations etc.Experience Details · Educational Detai,
        workexp : Procurement of billing the project and Project lead · The solution involves in creating dashboards and stories that depict different levels and stages.· The very first is a managerial dashboard to give quick overview of the Categories and their overview among different geographical areas, trends and comparisons using Map Charts, Pies, Stacked Bars, Scatter Plots etc.· The second one concentrates more on slicing and dicing the inventory and sales data using Dual Axis Charts, Various line charts, Waterfall charts etc.· The last one is a blend of Calculated Fields, Table Calculations and a bit of LODs to answer different types of the requirements of the client. Role : Tableau Developer Revenue Growth in % · Used Filters to know Department wise Sales and their Cost for Particular Periods and draft various charts using Show Me in Tableau Desktop.,
        education : null, 
        certtrainings : null, 
        gaps : null, 
        skilltags : null, 
        forreqid : FH-REQ-3, 
        uploadedbyuser : null
        }
    ]
}

Query code below --

Query workExQuery = SimpleQueryStringQuery.of(q -> q.query(finalQuery)
                    .fields(Arrays.asList("applications.workexp")).defaultOperator(Operator.And))._toQuery();
            queries.add(workExQuery);

            Query summaryQuery = SimpleQueryStringQuery.of(q -> q.query(finalQuery)
                    .fields(Arrays.asList("applications.summary")).defaultOperator(Operator.And))._toQuery();
            queries.add(summaryQuery);

Query 2 formation:

Query query = Query.of(q -> q.nested(p -> p.path("applications").query(b -> b.bool(bq -> bq.mustNot(queries)))));

Final Query esClient.search(s -> s.index("candidature").query(query), Someclass.class);

Query 1 formation :

Query query = Query.of(q -> q.nested(p -> p.path("applications").query(b -> b.bool(bq -> bq.must(queries)))));

Mappings for the field in java

@Field(type = FieldType.Keyword, name = "applnid")
    private String applnid;

    @Field(type = FieldType.Text, name = "summary")
    private String summary;

    @Field(type = FieldType.Text, name = "skills")
    private String skills;

    @Field(type = FieldType.Text, name = "employers")
    private String employers;

    @Field(type = FieldType.Integer, name = "totalexperienceinyears")
    private String totalexperienceinyears;

    @Field(type = FieldType.Text, name = "workexp")
    private String workexp;

    @Field(type = FieldType.Text, name = "education")
    private String education;

    @Field(type = FieldType.Text, name = "certtrainings")
    private String certtrainings;

    @Field(type = FieldType.Integer, name = "gaps")
    private String gaps;

    @Field(type = FieldType.Text, name = "skilltags")
    private String skilltags;
    
    @Field(type = FieldType.Keyword, name = "forreqid")
    private String forreqid;
    
    @Field(type = FieldType.Keyword, name = "uploadedbyuser")
    private String uploadedbyuser;

Mappings from elastic curl query

{
    "candidature": {
        "aliases": {},
        "mappings": {
            "properties": {
                "_class": {
                    "type": "keyword",
                    "index": false,
                    "doc_values": false
                },
                "applications": {
                    "type": "nested",
                    "include_in_parent": true,
                    "properties": {
                        "_class": {
                            "type": "keyword",
                            "index": false,
                            "doc_values": false
                        },
                        "applnid": {
                            "type": "keyword"
                        },
                        "certtrainings": {
                            "type": "text"
                        },
                        "education": {
                            "type": "text"
                        },
                        "employers": {
                            "type": "text"
                        },
                        "forreqid": {
                            "type": "keyword"
                        },
                        "gaps": {
                            "type": "integer"
                        },
                        "skills": {
                            "type": "text"
                        },
                        "skilltags": {
                            "type": "text"
                        },
                        "summary": {
                            "type": "text"
                        },
                        "totalexperienceinyears": {
                            "type": "integer"
                        },
                        "uploadedbyuser": {
                            "type": "keyword"
                        },
                        "workexp": {
                            "type": "text"
                        }
                    }
                },
                "candidateId": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "contactnumber": {
                    "type": "keyword"
                },
                "createdby": {
                    "type": "keyword"
                },
                "email": {
                    "type": "keyword"
                },
                "fororg": {
                    "type": "keyword"
                },
                "name": {
                    "type": "text"
                }
            }
        },
        "settings": {
            "index": {
                "routing": {
                    "allocation": {
                        "include": {
                            "_tier_preference": "data_content"
                        }
                    }
                },
                "refresh_interval": "1s",
                "number_of_shards": "1",
                "provided_name": "candidature",
                "creation_date": "1694872335334",
                "store": {
                    "type": "fs"
                },
                "number_of_replicas": "1",
                "uuid": "9cRKu-TLRsGfwuKn2UTbWg",
                "version": {
                    "created": "8080299"
                }
            }
        }
    }
}

Solution

  • In the simple_query_string query Or is treated as just a word. So your query is basically equivalent to searching for "Tableau" AND "OR" AND "workbooks" in the query_string query. None of the fields contain the word Or and as a result you don't get back anything.

    If your expectation were that Or in your query would work as a boolean operator you need to replace it with |. In other words your query should become

    (Tableau) | (Tableau) | (Tableau workbooks)

    See Simple Query Syntax for more information.