I have made a composite index as per the firebase docs
gcloud alpha firestore indexes composite create \
--project=[Insert Project Name Here]
--collection-group=elements --query-scope=COLLECTION \
--field-config=vector-config='{"dimension":"1536","flat": "{}"}',field-path=embedding
That works fine and I can see it clearly under gcloud firestore indexes composite describe [Insert Index Name Here]
Then I try to query the database using an onCall function.
export const vectorSearch = functions.https.onCall(async (data, context) => {
const { query, quantity } = data;
if (!query) {
throw new functions.https.HttpsError('failed-precondition', 'query is required');
}
if (!quantity) {
throw new functions.https.HttpsError('failed-precondition', 'quantity is required');
}
try {
console.log(`Creating embedding for ${query}`)
const embeddingResponse = await openai.embeddings.create({
model: "text-embedding-3-small",
input: query,
encoding_format: "float",
});
const embedding = embeddingResponse.data[0].embedding;
console.log(`Embedding:`, embedding)
const vectorQuery: VectorQuery = firestore.collection('elements').findNearest('embedding', FieldValue.vector(embedding), {
limit: quantity,
distanceMeasure: 'COSINE'
});
const snapshot: VectorQuerySnapshot = await vectorQuery.get();
console.log(`Snapshot:`, snapshot)
const formattedData: any = {};
snapshot.docs.forEach((doc) => {
formattedData[doc.id] = doc.data();
});
return { docs: formattedData };
} catch (error) {
console.error('Error querying database:', error);
throw new functions.https.HttpsError('internal', 'Error querying database');
}
});
When I call this function, I get no entries despite having documents under the relevant collection. I don't get any errors - just no documents appear in the response. However I should be seeing the number of documents I specified in the quantity
field. Why is this?
I was storing each embedding field as an array in my firestore database.
[0.1, 0.2, 0.3]
I should have been storing each embedding as a type of vector
.
FieldValue.vector([0.1, 0.2, 0.3])
Firebase ignored the fields that weren't a vector, so the index for the search was empty and therefore no documents were returned.