javascriptnode.jsstreamgithub-apigithub-copilot

How to get response of copilot LLM in github chat completions api


I'm building a Github Copilot extension.

However, I'm facing a issue in the extension. I'm using Copilot LLM for the extension.

Here's my code, which I copied from a example repo:


  // Use Copilot's LLM to generate a response to the user's messages, with
  // our extra system messages attached.
  const copilotLLMResponse = await fetch(
    "https://api.githubcopilot.com/chat/completions",
    {
      method: "POST",
      headers: {
        authorization: `Bearer ${tokenForUser}`,
        "content-type": "application/json",
      },
      body: JSON.stringify({
        messages,
        stream: true,
      }),
    }
  );

  const stream = Readable.from(copilotLLMResponse.body);

  stream.pipe(res);

This code perfectly streams the response to user, but now, I wanted to get the text myself.

I tried to get the text, but I failed.

I tried this guide, replacing all the vars:

import { Readable } from "stream";
const stream = Readable.from([Buffer.from("Hello, world!")]);
const text = await new Response(stream).text();
console.log(text); // "Hello, world!"

But I wasn't able to get the copilot response directly, and I was only able to get like:

data: {"choices":[{"finish_reason":"stop","index":0,"content_filter_offsets":{"check_offset":1896,"start_offset":1792,"end_
offset":1897},"content_filter_results":{"error":{"code":"","message":""},"hate":{"filtered":false,"severity":"safe"},"self_
harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"seve
rity":"safe"}},"delta":{"content":null}}],"created":1730025876,"id":"chatcmpl-someId","usage":{"comp
letion_tokens":22,"prompt_tokens":404,"total_tokens":426},"model":"gpt-3.5-turbo-0613"}

(it was being printed a few times)

To avoid it as being flagged as a XY Question

I want to inject some pieces of documentation into copilot(which it'll request in the message), so it gives a valid response to the user.


Solution

  • The issue was that in fetch request, I was actually setting stream to true.

    after setting it to false, I was able to get the json & actual response