I want to run the DeepSeek-V3 model inference using the Hugging-Face Transformer library (>= v4.51.0).
I read that you can do the following to do that (download the model and run it)
from transformers import pipeline
messages = [
{"role": "user", "content": "Who are you?"},
]
pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-R1", trust_remote_code=True)
pipe(messages)
My issue is that I already downloaded the DeepSeek-V3 hugging-face repository separately, and I just want to tell the Transformer where it is on my local machine, so that it can run the inference.
The model repository is thus not (or not necessarily) in the Hugging-Face cache directory (it can be anywhere on the local machine). When loading the model, I want to provide the path which specifically points to the model's repository on the local machine.
How can I achieve that?
Since you said you downloaded the model already from Huggingface, I assume you downloaded all of the related Huggingface files including the JSON files in the repo that describe the model for loading. In this case, the pipeline function can easily take a filesystem path in the model parameter instead of a model name.
For example, if you downloaded the files to the folder /my-models/deepseek-r1, you just need to load it this way
pipe = pipeline("text-generation", model="/my-models/deepseek-r1", trust_remote_code=True)
The pipeline function will try to load from the filesystem, and if not found, it will try to look for a model with that name on the Hub.