I'm having trouble getting function call working with OpenAI Agent SDK and Ollama. Suggestions/Solutions would be of great help. Thank you.
Using UV, python-12 and added "openai>=1.68.2" and "openai-agents>=0.0.15" dependencies to the project.
Below is my code:
from dotenv import load_dotenv
from agents import Agent, Runner, AsyncOpenAI, OpenAIChatCompletionsModel, function_tool
import asyncio
load_dotenv(override=True)
ollama_url = 'http://localhost:11434/v1'
ollama_api_key = 'dummy'
model = 'qwen3:4b'
@function_tool
def math_tool(a: int, b: int) -> int:
""" Send two integers and this returns their sum """
print(f"Adding {a} and {b}")
return a + b
async def main():
ollama_client = AsyncOpenAI(base_url=ollama_url, api_key=ollama_api_key)
ollama_qwen3_4b_model = OpenAIChatCompletionsModel(model=model, openai_client=ollama_client)
simple_agent = Agent(name="simple_agent", instructions="You are a simple agent that can perform basic tasks", tools=[math_tool], model=ollama_qwen3_4b_model)
output = await Runner.run(simple_agent, "What is 2 + 2?")
print(output)
asyncio.run(main())
It errors out, I assume below is the prime error.
File "/Users/RandomName/.local/share/uv/python/cpython-3.12.11-macos-aarch64-none/lib/python3.12/typing.py", line 501, in __call__
raise TypeError(f"Cannot instantiate {self!r}")
TypeError: Cannot instantiate typing.Union
This is the full trace when I run uv run ollama_simple.py
Traceback (most recent call last):
File "/Users/RandomName/Documents/ai/github/openai_agent_learning/ollama_agent_tool_simple/ollama_simple.py", line 33, in <module>
asyncio.run(main())
File "/Users/RandomName/.local/share/uv/python/cpython-3.12.11-macos-aarch64-none/lib/python3.12/asyncio/runners.py", line 195, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/Users/RandomName/.local/share/uv/python/cpython-3.12.11-macos-aarch64-none/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/RandomName/.local/share/uv/python/cpython-3.12.11-macos-aarch64-none/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/Users/RandomName/Documents/ai/github/openai_agent_learning/ollama_agent_tool_simple/ollama_simple.py", line 30, in main
output = await Runner.run(simple_agent, "What is 2 + 2?")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/RandomName/Documents/ai/github/openai_agent_learning/ollama_agent_tool_simple/.venv/lib/python3.12/site-packages/agents/run.py", line 206, in run
return await runner.run(
^^^^^^^^^^^^^^^^^
File "/Users/RandomName/Documents/ai/github/openai_agent_learning/ollama_agent_tool_simple/.venv/lib/python3.12/site-packages/agents/run.py", line 434, in run
turn_result = await self._run_single_turn(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/RandomName/Documents/ai/github/openai_agent_learning/ollama_agent_tool_simple/.venv/lib/python3.12/site-packages/agents/run.py", line 956, in _run_single_turn
new_response = await cls._get_new_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/RandomName/Documents/ai/github/openai_agent_learning/ollama_agent_tool_simple/.venv/lib/python3.12/site-packages/agents/run.py", line 1168, in _get_new_response
new_response = await model.get_response(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/RandomName/Documents/ai/github/openai_agent_learning/ollama_agent_tool_simple/.venv/lib/python3.12/site-packages/agents/models/openai_chatcompletions.py", line 66, in get_response
response = await self._fetch_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/RandomName/Documents/ai/github/openai_agent_learning/ollama_agent_tool_simple/.venv/lib/python3.12/site-packages/agents/models/openai_chatcompletions.py", line 228, in _fetch_response
converted_messages = Converter.items_to_messages(input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/RandomName/Documents/ai/github/openai_agent_learning/ollama_agent_tool_simple/.venv/lib/python3.12/site-packages/agents/models/chatcmpl_converter.py", line 443, in items_to_messages
new_tool_call = ChatCompletionMessageToolCallParam(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/RandomName/.local/share/uv/python/cpython-3.12.11-macos-aarch64-none/lib/python3.12/typing.py", line 1184, in __call__
result = self.__origin__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/RandomName/.local/share/uv/python/cpython-3.12.11-macos-aarch64-none/lib/python3.12/typing.py", line 501, in __call__
raise TypeError(f"Cannot instantiate {self!r}")
TypeError: Cannot instantiate typing.Union
The code is trying to do:
new_tool_call = ChatCompletionMessageToolCallParam(
id=file_search["id"],
type="function",
function={
"name": "file_search_call",
"arguments": json.dumps(
{
"queries": file_search.get("queries", []),
"status": file_search.get("status"),
}
),
},
)
Where:
ChatCompletionMessageToolCallParam: TypeAlias = Union[
ChatCompletionMessageFunctionToolCallParam, ChatCompletionMessageCustomToolCallParam
]
Where the two types in the union are pydantic base models.
The code is trying to use an initialiser call to a type, similar to:
from typing import Union
Foo = Union[int, str]
Foo("hello")
Traceback (most recent call last):
...
raise TypeError(f"Cannot instantiate {self!r}")
TypeError: Cannot instantiate typing.Union
So the code in the agent module is just wrong for the most recent version of the openai module.
According to this github issue, you should downgrade the openai module to <1.99.0