"openai.BadRequestError: Invalid role" error while using Ollama + LiteLLM + Autogen

Introduction

I am running an ollama server with LiteLLM proxy. I aim to design a group chat using Autogen. The designated agents are:

• User_Proxy: ask question
• SQL_Generator: generate SQL
• SQL_Runner: run SQL (tool calling)
• Result_Validator: validate result

Problems

1. The group chat works fine up until SQL_Runner returns the data from DB. But when the next speaker is expected to be the Result_Validator, the SQL_Generator is assigned as next speaker, although I explicitly state in the system message that SQL_Runner should pass the result to Result_Validator.

2. For some reason, SQL_Generator agent's role remains as tool following the activity of SQL_Runner instead of one of [system, user, assistant] and it leads to this error:

***** Response from calling tool (call_6493f510-d318-4e85-b552-8963bece5fcb) *****
[
    // data from DB
]
**********************************************************************************
Next speaker: SQL_Generator

// error trace

openai.BadRequestError: Error code: 400 - {'error': {'message': 'invalid role: tool, role must be one of [system, user, assistant]', 'type': 'api_error', 'param': None, 'code': None}}

Code

LLM Configs

llama3_config_list = [
    {
        "model": "llama3:latest",
        "base_url": f"{OLLAMA_BASE_URL}:{OLLAMA_PORT}/v1",
        "api_key": "ollama",
    }
]

litellm_config_list = [
    {
        "model": "NotRequired",
        "api_key": "NotRequired",
        "base_url": f"{LITELLM_BASE_URL}:{LITELLM_PORT}",
        "price": [0, 0],
    }
]

local_llm_config = {"config_list": litellm_config_list, "cache_seed": None}

GroupChat

user_proxy = autogen.UserProxyAgent(
    name=Role.USER_PROXY,
    system_message=Prompt.USER_PROXY,
    human_input_mode="NEVER",
    is_termination_msg=is_termination_msg,
    code_execution_config={"use_docker": False, "work_dir": "test"},
)

sql_generator = autogen.AssistantAgent(
    name=Role.SQL_GENERATOR,
    system_message=Prompt.SQL_GENERATOR,
    llm_config=get_custom_llm_config(config_list=llama3_config_list),
)

sql_runner = autogen.AssistantAgent(
    name=Role.SQL_RUNNER,
    system_message=Prompt.SQL_RUNNER,
    llm_config=local_llm_config,
)

@user_proxy.register_for_execution()
@sql_runner.register_for_llm(description="SQL query runner")
def run_sql(sql: Annotated[str, "SQL query string to run"]) -> str:
    with PostgresManager() as db:
        db.get_rds_connection()
        return db.run_sql(sql)

result_validator = autogen.AssistantAgent(
    name=Role.RESULT_VALIDATOR,
    system_message=Prompt.RESULT_VALIDATOR,
    llm_config=get_custom_llm_config(config_list=llama3_config_list)
)

groupchat = autogen.GroupChat(
    agents=[
        user_proxy,
        sql_generator,
        sql_runner,
        result_validator,
    ],
    messages=[],
)

manager = autogen.GroupChatManager(groupchat=groupchat)

user_proxy.initiate_chat(manager, message=question)

For the problem number 1, I have already tried to set autogen.GroupChat's speaker_selection_method to "round_robin" but the result did not change at all.

For the problem number 2, I am guessing that it might be overcome if I can somehow prevent agent role from being cached, but I cannot really figure out how.

I was encountering this error since my ollama version did not support tool calling.

After upgrading ollama server version (to 0.4.2) now the role "tool" is available.