Unexpected value passed to LangChain Tool argument

I'm new to LLMs, AI app development, and LangChain. I'm trying to create a simple example tool that creates new user accounts in a hypothetical application when instructed to do so via a user prompt. The llm being used is llama3.1:8b via Ollama.

So far what I've written works, but it's very unreliable. The reason why it's unreliable is because when LangChain calls on my tool, it provides unexpected/inconsistent values to the user creation tool's single username argument.

Sometime the argument will be a proper username and other times it will be a username with the value "username=" prefixed to the username (eg: "username=jDoe" rather than simply "jdoe").

Also, if I ask for multiple users to be created, sometimes langchain will correctly invoke the tool multiple times while other times, it will invoke the tool once with a string in the format of an array (eg: "['jDoe','jSmith']")

What I've just explained is my problem. My questions to everyone are:

1. Is the issue I'm encountering due to the limitations of LangChain or the Llama3.1:8b model that I'm using? Or is the issue something else?
2. Is there a way to get LangChain to more reliably call my user creation tool with a correctly formatted username?
3. Are there are other useful tips/recommendations that you can provide for a beginner like me?

Below is my code:

from dotenv import load_dotenv
from langchain.agents import AgentExecutor, create_react_agent
from langchain.tools import Tool
from langchain_core.prompts import PromptTemplate
from langchain_ollama.chat_models import ChatOllama

load_dotenv()


# Define the tool to create a user account
mock_user_db = ["jDoe", "jRogers", "jsmith"]


def create_user_tool(username: str):
    print("USERNAME PROVIDED FOR CREATION: " + username)
    if username in mock_user_db:
        return f"User {username} already exists."
    mock_user_db.append(username)
    return f"User {username} created successfully."


# Define the tool to delete a user account
def delete_user_tool(username: str):
    print("USERNAME PROVIDED FOR DELETION: " + username)
    if username not in mock_user_db:
        return f"User {username} does not exist."
    
    mock_user_db.remove(username)
    return f"User {username} deleted successfully."


def list_users_tool(ignore) -> list:
    return mock_user_db


# Wrap these functions as LangChain Tools
create_user = Tool(
    name="Create User",
    func=create_user_tool,
    description="Creates a new user account in the company HR system."
)

delete_user = Tool(
    name="Delete User",
    func=delete_user_tool,
    description="Deletes an existing user account in company HR system."
)

list_users = Tool(
    name="List Users",
    func=list_users_tool,
    description="Lists all user accounts in company HR system."
)

# Initialize the language model
llm = ChatOllama(model="llama3.1:latest", temperature=0)

# Create the agent using the tools
tools = [create_user, delete_user, list_users]

# Get the prompt to use
#prompt = hub.pull("hwchase17/react") # Does not work with ollama/llama3:8b
prompt = hub.pull("hwchase17/react-chat") # Kinda works with ollama/llama3:8b

agent = create_react_agent(llm, tools, prompt)

# Create an agent executor by passing in the agent and tools
agent_executor = AgentExecutor(agent=agent, tools=tools, handle_parsing_errors=True)

print(agent_executor.invoke({"input": "Please introduce yourself."})['output'])

while True:
    user_prompt = input("PROMPT: ")
    agent_response = agent_executor.invoke({"input": user_prompt})
    print(agent_response['output'])

Prompt engineering (what you are attempting here), is far from an exact science. However, there are ways you can clarify the schema of the tool.

One example (from their docs) is getting it to parse your docstrings:

@tool(parse_docstring=True)
def create_user(username: str):
    """Creates a user

        Args:
            username: username of the user to be created. The exact string of the username, no longer than 20 characters long
    """
    ... # Rest of your code here

See docs here

But even more reliable would be to create your schema with Pydantic (great tool in general), again, from their docs:

class create_user(BaseModel):
    """Creates a user"""

    username: str = Field(..., description="username of the user to be created. The exact string of the username, no longer than 20 characters long"

In general, the more detail you provide, regarding the shape and nature of the tools and the data, the better results you can expect.

You may also want to consider setting your temperature to 0, so you get repeatable responses for any given prompt, which should help with debugging, but you need to test with a higher range of prompts to ensure reliable behaviour