pythonpytorchpipmistral-7b

Impossible to install mistral-inference python package

I want to install the python package 'mistral-inference'.

My computer has a GPU and is using windows 11.
Python 3.13.2
pip 24.3.1

I created a new virtual environnment. Then I ran 'pip install mistral-inference'. I got the following error:

Collecting xformers>=0.0.24 (from mistral-inference)
  Using cached xformers-0.0.29.post3.tar.gz (8.5 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  ...

ModuleNotFoundError: No module named 'torch'

I then tried to install 'torch' in my venv beforehand, I tried many versions of it (CPU, CUDA 11.8, CUDA 12.4, CUDA 12.6) but I always get the same error.

I also tried to download the source code of 'mistral-inference' from github, but got here the same kind of problem:

Collecting jnius==1.1.0 (from -r .\requirements.txt (line 16))
  Using cached jnius-1.1.0.tar.gz (28 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  ...

ModuleNotFoundError: No module named 'Cython'

Same I tried to install first Cython, I tried the latest version, I tried version 0.29.37, always the same error.

What should I do?

Solution

  • Faced the same issue with this Python version. Python 3.13 is not fully supported by many libraries yet, including torch, xformers, and jnius! I strongly recommend downgrading to Python 3.10 or 3.11, as these versions have better compatibility.

    Some steps I used to run it in virtual enviroment after I set Python to 3.11 in PATH:

    python -m venv mistral_env
mistral_env\Scripts\activate

    This way the following installs have completed successfully in the virtual enviroment:

    pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install cython
pip install numpy scipy sentencepiece
pip install xformers
pip install mistral-inference