pythonwhitespacelarge-language-modelollamamistral-7b

Mistral7b response starts with an extra leading space when streamed with Ollama


When I stream the response of mistral7b LLM with Ollama, it has an extra space to the left on the very first streamed chunk. Below is my code:

import ollama

stream = ollama.chat(
  model='mistral',
  messages=[{'role': 'user', 'content': 'Name an engineer that passes the vibe check'}],
  stream=True
)

for chunk in stream:
  print(chunk['message']['content'], end='', flush=True)

The output looks like this:

$ python3 test.py 
 Elon Musk, the CEO of SpaceX and Tesla, is an engineer who seems to pass the "vibe check." He is known for his innovative ideas in renewable energy, space travel, and transportation. However, it's important to remember that personality and vibes can be subjective, so not everyone may agree with this assessment. Additionally, Musk's public image should not overshadow the contributions of countless other engineers who are equally impressive but less well-known.

Note that very first empty space before the letter "E". How do I remove it?


Solution

  • Use lstrip() to remove the white space.

    import ollama
    
    stream = ollama.chat(
      model='mistral',
      messages=[{'role': 'user', 'content': 'Name an engineer that passes the vibe check'}],
      stream=True
    )
    first = True
    for chunk in stream:
        if first:
            chunk = chunk['message']['content'].lstrip()
            print(chunk)
            first = False
        else:
            chunk = chunk['message']['content']
            print(chunk)