pythonnlphuggingface-transformerstransformer-modelllama

cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub'

I've been using LLAMA 2 for research for a few months now and I import as follows:

from transformers import AutoModelForCausalLM, AutoTokenizer
device = torch.device("cuda")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf",token = "token_key",torch_dtype="auto")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-chat-hf",token = "token_key", torch_dtype="auto", load_in_4bit=True)

It has always worked. However, today it is showing the following error: RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback): Failed to import transformers.generation.utils because of the following error (look up to see its traceback): cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub' (/opt/conda/lib/python3.10/site-packages/huggingface_hub/init.py)

Recreated the Hugging Face token, but it didn't work. I am using Google Colab and Kaggle Notebook.

Solution

  • The error you're encountering is due to the split_torch_state_dict_into_shards function not being available in huggingface-hub version < 0.23.0.

    This function is included starting from version 0.23.0.

    To resolve this issue, update the huggingface-hub library to version 0.23.0 or later

    here is a git link: https://github.com/run-llama/llama_index/discussions/14605