I was reading the docs of GenKit. Does the history that I pass to the model counts as input tokens for the pricing of the model? Or is the input token cost related only to the "prompt" field?
let history: MessageData[] = [
{ role: 'system', content: [{ text: 'Talk like a pirate.' }] },
];
let response = await generate({
model: geminiPro,
prompt: "How do you say 'dog' in French?",
history,
});
history = response.toHistory();
Docs I was reading: https://firebase.google.com/docs/genkit/models
Yes history counts in tokens.
The models available through the Gemini API have context windows that are measured in tokens. The context window defines how much input you can provide and how much output the model can generate.
How much you send in your context windows defines what you will be charged.
Calling count_tokens with the chat history returns a total_tokens value that is the token count of the text from each role in the chat.
To understand how big your next conversational turn will be, you need to append it to the history when you call count_tokens.
See: tokens see: Count tokens and billable characters
Note: there is no real documentation that states "history" on firebase gemini. I have sent out a bunch of messages to google to get that fixed.