google-cloud-platformlarge-language-modelgoogle-gemini

Does gemini provide async prompting like anthropic and openai?

I've been using gemini for the summarization of multi-page pdfs and using the chunking technique, but as the parallel processing is being managed through the code, it's a bit slower. I believe Async API will solve my problem, but I couldn't find it in the documentation.

Solution

  • Yes, Gemini GenerativeModel has a native async API implementation - generate_content_async. You can easily wrap it under async client framework such as Python asyncio.

    This blog from Paul Balm went in detail with sample code on how to prompting Gemini async natively.