pythonhuggingface-transformershuggingface

How to get the code of the hugging face models?


There is a simple way to download a model from hugging face,

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
model = AutoModel.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")

use it, train it and even replace the layers as long as they take the same input (shapes) and give the same output.

Since it is based on pytorch (in this case), you get a nice view of the layers once you print it out, but you don't know the forward methods and other methods used in pytorchs model class.

Is there a way to get the entire model in a .py file (as a nn.Module) class like it must have been initially used for training (and the weights e.g. as a .pt file)?


Solution

  • You can go to the model page on Hugging Face, click on "Files and versions," and check the config.py file. In the architectures field, you will find that the model "nreimers/MiniLM-L6-H384-uncased" is based on the "BertModel".

    For example, you can refer to the config.json here: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/blob/main/config.json#L4

    To view the class definition and understand its full implementation, you can visit Hugging Face’s GitHub repository at the following link: https://github.com/huggingface/transformers/blob/v4.52.3/src/transformers/models/bert/modeling_bert.py

    Additionally, for more detailed information about the BERT model, you can check the official Hugging Face documentation here: https://huggingface.co/docs/transformers/model_doc/bert

    This documentation will give you a better understanding of the model's structure, parameters, and usage.