node.jsgoogle-drive-apiopenai-apisupabasedeno

Send Google Drive API File Download to OpenAI

I'm trying to send a file from Google Drive to OpenAI API via chat completion

I've already authenticated and I'm able to list files and get the file in question. The Google API returns the data as a Blob.

The problem I'm running into is that when I try to convert the Blob to a base64 encoded string, I get a RangeError or DOMException from the Supabase Edge Function

Here's a snippet from the code I have right now (post auth and files.get):

const fileDownload = await driveApi.files.get({
  fileId: 'file_id_string',
  alt: 'media',
});
console.log('file download', fileDownload.data);

const blob: Blob = fileDownload.data;
const arrayBuffer = await blob.arrayBuffer();
const uint8Array = new Uint8Array(arrayBuffer);

// Attempt 1
const charCodeString = String.fromCharCode(...uint8Array); // <-- Fails here
const base64String = btoa(charCodeString);

// Attempt 2
const decodedString = new TextDecoder().decode(uint8Array);
const base64String = btoa(decodedString); // <-- Fails here

// Uncomment once I get the base64String right
// await openai.chat.completions.create({
//   model: 'gpt-4o-mini',
//   response_format: 'json_object',
//   messages: [
//     { role: 'user', content: [{ url: base64String, type: 'image_url' }] },
//   ],
// });

# Attempt 1 error message
 RangeError: Maximum call stack size exceeded
    at Object.handler (file:///repos/supabase-personal/supabase/functions/openai/index.ts:32:38)
    at Object.runMicrotasks (ext:core/01_core.js:642:26)
    at processTicksAndRejections (ext:deno_node/_next_tick.ts:39:10)
    at runNextTicks (ext:deno_node/_next_tick.ts:48:3)
    at eventLoopTick (ext:core/01_core.js:175:21)
    at async respond (ext:sb_core_main_js/js/http.js:163:14)

# Attempt 2 error message
DOMException: The string to be encoded contains characters outside of the Latin1 range.
    at new DOMException (ext:deno_web/01_dom_exception.js:116:20)
    at btoa (ext:deno_web/05_base64.js:52:13)
    at Object.handler (file:///repos/supabase-personal/supabase/functions/openai/index.ts:37:26)
    at Object.runMicrotasks (ext:core/01_core.js:642:26)
    at processTicksAndRejections (ext:deno_node/_next_tick.ts:39:10)
    at runNextTicks (ext:deno_node/_next_tick.ts:48:3)
    at eventLoopTick (ext:core/01_core.js:175:21)
    at async respond (ext:sb_core_main_js/js/http.js:163:14)

I really appreciate any help that can be given!

I'm expecting, based on the OpenAI documentation, to convert the file to base64 to send to the chat completion endpoint. Is there an easier, or better way to do this?

Solution

  • I believe your goal is as follows.

    In this case, how about the following modification?

    Modified script:

    const fileId = "file_id_string"; // Please set your file ID.

const fileDownload = await driveApi.files.get(
  { fileId, alt: "media" },
  { responseType: "stream" }
);
let buf = [];
fileDownload.data.on("data", (e) => buf.push(e));
fileDownload.data.on("end", () => {
  const buffer = Buffer.concat(buf);
  const base64String = buffer.toString("base64");
  console.log(base64String);
});

    Or, when responseType: "arraybuffer" is used, how about the following modification?

    const fileId = "file_id_string"; // Please set your file ID.

const fileDownload = await driveApi.files.get(
  { fileId, alt: "media" },
  { responseType: "arraybuffer" }
);
const base64String = Buffer.from(fileDownload.data).toString("base64");
console.log(base64String);

    Reference: