[SOLVED] ollama.generate raises model not found error: "hf.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF"

ollama.generate raises model not found error: "hf.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF"

I'm trying to run a Python script that uses the ollama library to generate responses from a custom LLM model. My code attempts to call ollama.generate() using the following model name:

chosen_model = 'hf.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF'

def extract_keyword(prompt):
    response = ollama.generate(
        model=chosen_model,
        prompt=f"Identify the product/item in {prompt}. ..."
    )
    return response.get('response', '').strip()

I have ollama installed and running locally, and other standard models work fine.

Is 'hf.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF' a valid model name for use with ollama?
Do I need to pull the model manually using ollama pull or convert it in some way?
How can I verify the availability or compatibility of a model with ollama?

Error

Traceback (most recent call last):
File "/home/jas/Desktop/WNE3/Old/updatedPromptEnhancer.py", line 113, in <module>
main()
File "/home/jas/Desktop/WNE3/Old/updatedPromptEnhancer.py", line 80, in main
keyword = extract_keyword(user_input)
File "/home/jas/Desktop/WNE3/Old/updatedPromptEnhancer.py", line 33, in extract_keyword
response = ollama.generate(
File "/home/jas/anaconda3/lib/python3.11/site-packages/ollama/_client.py", line 242, in generate
return self._request(
File "/home/jas/anaconda3/lib/python3.11/site-packages/ollama/_client.py", line 178, in _request
return cls(**self._request_raw(*args, **kwargs)).json()
File "/home/jas/anaconda3/lib/python3.11/site-packages/ollama/_client.py", line 122, in _request_raw
raise ResponseError(e.response.text, e.response.status_code) from None
ollama._types.ResponseError: model 'hf.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF' not found (status code: 404)

Link to code file:https://github.com/jahnvisikligar/Python-concepts/blob/main/updatedPromptEnhancer.py

Any help to resolve this would be appreciated!

Solution

You must always have the model available on the Ollama server when calling from API. The Ollama CLI interface would pull it for you, but it won't do the same for the API. You have to pull it first.

Since you are using Ollama Python Library, you can use the following to list all the models available on your server and pull the models you want to the server. You always need to pull before you use it for API access.

List all models

ollama.list()

Pull model

ollama.pull('llama3.2')

Check Ollama API doc for more details.