pythonllamaollama

Running ollama on kaggle

I downloaded ollama on a kaggle notebook (linux). I want to interact with it using a python script. On following the instructions on the github repo and running: ollama run llama3 I got the output: Error: could not connect to ollama app, is it running?.

It appears that I need to run ollama serve before running llama3. However the entire main thread is taken up by ollama serve so you cannot run anything else after it.

The work arounds I tried:

Full logs of second method:

TypeError                                 Traceback (most recent call last)
Cell In[29], line 1
----> 1 subprocess.run('ollama', 'serve')

File /opt/conda/lib/python3.10/subprocess.py:503, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    500     kwargs['stdout'] = PIPE
    501     kwargs['stderr'] = PIPE
--> 503 with Popen(*popenargs, **kwargs) as process:
    504     try:
    505         stdout, stderr = process.communicate(input, timeout=timeout)

File /opt/conda/lib/python3.10/subprocess.py:780, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize)
    778     bufsize = -1  # Restore default
    779 if not isinstance(bufsize, int):
--> 780     raise TypeError("bufsize must be an integer")
    782 if pipesize is None:
    783     pipesize = -1  # Restore default

TypeError: bufsize must be an integer

I chose ollama because the setup was simple (just running a single command). I am ok with using another method besides ollama, however I want to run it on python without much setup as it is running on kaggle

Solution

  • This is what I did to get it to work:

    #Download ollama
!curl -fsSL https://ollama.com/install.sh | sh
import subprocess
process = subprocess.Popen("ollama serve", shell=True) #runs on a different thread
#Download model
!ollama pull llama3
!pip install ollama
import ollama

#Then everytime you want to chat
response = ollama.chat(model='llama3', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response['message']['content'])