I try to implement a search engine using elasticsearch. For that i created an index with a mapping that contains dense_vector fields. PUT {{elastic uri}}/{{index}}
{
"mappings": {
"properties": {
...
"title_embeddings": {
"type": "dense_vector",
"index": true,
"similarity": "cosine",
"dims": 1024,
"index_options": {
"type": "hnsw",
"ef_construction": 100,
"m": 16
}
},
"title": {
"type": "text"
},
...
}
}
}
The problem is that when i retrieve it using the API, the field is of type "float" instead of dense_vector.
GET {{elastic-uri}}/{{index}}/_search
{
"{{index}}": {
"mappings": {
"properties": {
...
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"title_embeddings": {
"type": "float"
}
...
}
}
}
}
Also i can't use the KNN search on it since it's not the correct type :
GET {{elastic-uri}}/{{index}}/_search
{
"knn": {
"field": "title_embeddings",
"query_vector": [0.1, 3.2, 2.1],
"k": 2,
"num_candidates": 100
}
}
Error : failed to create query: [knn] queries are only supported on [dense_vector] fields
I was indexing my data with the following code :
es.index(
index='{{index}}',
document=json.dumps(page.__dict__),
error_trace=True
)
And with this new code it works :
es.index(
index='{{index}}',
document={
"id": page.id,
"title": page.title,
"title_embeddings": page.title_embeddings,
...
},
error_trace=True
)