I'm following the instructions of 5. Call the Gemini API with images from Automate Google Workspace tasks with the Gemini API. When running testGeminiVision, I got a truncated error message suggesting using muteHttpExceptions.

I modified the code by adding 'muteHttpExceptions': true to the options variable declaration in the callGeminiProVision function. Now I got the following in the Execution logs:

Provide a fun fact about this object. {
  "error": {
    "code": 404,
    "message": "Gemini 1.0 Pro Vision has been deprecated on July 12, 2024. Consider switching to different model, for example gemini-1.5-flash.",
    "status": "NOT_FOUND"
  }
}

Googling the message, I found an issue on a GitHub repository: https://github.com/OthersideAI/self-operating-computer/issues/204, but it hasn't a fix yet.

I also searched for release notes and on the Gemini documentation, but I needed help fixing the Codelab code. I have found suggestions to switch to Google AI Vertex, but doing this requires significant changes.

Minimal Complete Example

Code adapted from the referred Codelab. I removed the parts that are not required to reproduce the issue that I'm facing. To run this code, you must get a Gemini API key and add a script property using the GOOGLE_API_KEY as the property name. The referred Codelab has detailed instructions for doing this.

Once the Apps Script project setup is complete in the Google Apps Script Editor, select testGeminiVision, then click Run.

const properties = PropertiesService.getScriptProperties().getProperties();
const geminiApiKey = properties['GOOGLE_API_KEY'];
const geminiProVisionEndpoint = `https://generativelanguage.googleapis.com/v1beta/models/gemini-1.0-pro-vision-latest:generateContent?key=${geminiApiKey}`;

function callGeminiProVision(prompt, image, temperature = 0) {
    const imageData = Utilities.base64Encode(image.getAs('image/png').getBytes());

    const payload = {
        "contents": [
            {
                "parts": [
                    {
                        "text": prompt
                    },
                    {
                        "inlineData": {
                            "mimeType": "image/png",
                            "data": imageData
                        }
                    }
                ]
            }
        ],
        "generationConfig": {
            "temperature": temperature,
        },
    };
    
    const options = {
        'method': 'post',
        'contentType': 'application/json',
        'payload': JSON.stringify(payload),
        'muteHttpExceptions': true
    };
    
    const response = UrlFetchApp.fetch(geminiProVisionEndpoint, options);
    if (response.getResponseCode() !== 200) {
        return response.getContentText();
    }
    const data = JSON.parse(response);
    const content = data["candidates"][0]["content"]["parts"][0]["text"];
    return content;

}

function testGeminiVision() {
    const prompt = "Provide a fun fact about this object.";
    const image = UrlFetchApp.fetch('https://storage.googleapis.com/generativeai-downloads/images/instrument.jpg').getBlob();
    const output = callGeminiProVision(prompt, image);
    console.log(prompt, output);
}

TLDR:

Replace models/gemini-1.0-pro-vision-latest with models/gemini-1.5-flash-latest.

From

const geminiProVisionEndpoint = `https://generativelanguage.googleapis.com/v1beta/models/gemini-1.0-pro-vision-latest:generateContent?key=${geminiApiKey}`;

To

const geminiProVisionEndpoint = `https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent?key=${geminiApiKey}`;