javascriptollama

Ollama API seems to ignore messages


I have setup a simple webinterface to communicate to the Ollama API and output the responses in a chat window. When requesting the model normally like this:

{
    "model": "llama3",
    "prompt": "Hello world"
}

It returns the standard response stream. When i do it like this however:

{
        "model": "llama3",
        "messages": [
            {
                "role": "user",
                "content": "why is the sky blue?"
            },
            {
                "role": "assistant",
                "content": "due to rayleigh scattering."
            },
            {
                "role": "user",
                "content": "how is that different than mie scattering?"
            }
        ]
    }

(This is just the example from the documentation)

It returns the message like it's just the "load" request which is an empty string. When using cURL it works just fine. What am i missing here?

This is the code i use to request:

let url = "http://localhost:11434/api/generate";
    const data = {
        "model": "llama3",
        "messages": [
            {
                "role": "user",
                "content": "why is the sky blue?"
            },
            {
                "role": "assistant",
                "content": "due to rayleigh scattering."
            },
            {
                "role": "user",
                "content": "how is that different than mie scattering?"
            }
        ]
    };

    console.log(data);

    const requestOptions = {
        method: "POST",
        headers: {
            'Content-Type': 'application/json',
        },
        body: JSON.stringify(data),
    };

    fetch(url, requestOptions)
        .then(res => res.body)
        .then(rb => {
            const reader = rb.getReader();
            
            return new ReadableStream({
                start(controller) {
                    function push() {
                        reader.read().then(async ({done, value}) => {
                            if (done) {
                                chatbox.innerHTML += "</p>";
                                console.log("done", done);
                                controller.close();
                                return;
                            }
                            // Fetch the individual words
                            await controller.enqueue(value);
                            let json = JSON.parse(new TextDecoder().decode(value));
                            chatbox.innerHTML += json.response;
                            push();
                        });
                    }
                    
                    push();
                }
            });
        })
        .then(stream => 
            new Response(stream, { headers: { "Content-Type": "text/html" } }).text()
        )
        .then(result => {
            console.log("Result:" + result);
        });

Solution

  • make sure to use /api/chat for message-based prompts. It seems like you may be using /api/generate in your example.