I have created a chatbot in Flutter that uses the function calling mechanism of Vertex AI to query a database. Some of the function calls return several thousand data points which are then sent to the model for processing.
I notice that in a typical conversation, the same function call will be made repeatedly over several interactions. This increases the cost of the interaction significantly and also appears to be redundant.
Is it necessary to keep feeding Vertex AI these data points each time a query is relevant to that function call? If not, what is the best way to handle it? In other words, can I handle this via a system instruction (i.e. "Don't call the same function twice in a conversation, just remember the results of the last call") or should I modify the function call in some way?
Currently no documentation is available in function calling mechanism of Vertex AI on Gemini to cache the results and use it when the same function call repeat . Now it’s in preview stage , may be feature available in the future .
If you want that feature you can open a new Feature request as per your requirements on the public issue tracker describing your issue and vote [+1] and Eng team will look on the feature for future implementation