huggingfacellamafine-tuninghuggingface-hubllama3.1

Error when pushing Llama3.1 7B fine-tuned model to Huggingface

I'm having an issue pushing a fine-tuned Llama 3.1 model to Huggingface, getting the error below. All of the literature that I've read suggests that the code below that I'm using to push is correct, and the model exists on my page, there are just no files or model card associated with it.

model.push_to_hub(NEW_MODEL, token=token, max_shard_size="10GB")

The loading of said model code is below, done before model.push:

tokenizer = AutoTokenizer.from_pretrained(NEW_MODEL)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    torch_dtype=torch.float16,
    device_map="auto",
)

model.resize_token_embeddings(len(tokenizer), pad_to_multiple_of=8)
model = PeftModel.from_pretrained(model, NEW_MODEL)
model = model.merge_and_unload()

Does anyone have a word of advice here?

Cheers!

---------------------------------------------------------------------------
IsADirectoryError                         Traceback (most recent call last)
<ipython-input-86-cac150fe4ba1> in <cell line: 1>()
----> 1 model.push_to_hub(NEW_MODEL, token=token, max_shard_size="10GB")

3 frames
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py in push_to_hub(self, repo_id, use_temp_dir, commit_message, private, token, max_shard_size, create_pr, safe_serialization, revision, commit_description, tags, **deprecated_kwargs)
    917 
    918         # Create a new empty model card and eventually tag it
--> 919         model_card = create_and_tag_model_card(
    920             repo_id, tags, token=token, ignore_metadata_errors=ignore_metadata_errors
    921         )

/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py in create_and_tag_model_card(repo_id, tags, token, ignore_metadata_errors)
   1172     try:
   1173         # Check if the model card is present on the remote repo
-> 1174         model_card = ModelCard.load(repo_id, token=token, ignore_metadata_errors=ignore_metadata_errors)
   1175     except EntryNotFoundError:
   1176         # Otherwise create a simple model card from template

/usr/local/lib/python3.10/dist-packages/huggingface_hub/repocard.py in load(cls, repo_id_or_path, repo_type, token, ignore_metadata_errors)
    185 
    186         # Preserve newlines in the existing file.
--> 187         with card_path.open(mode="r", newline="", encoding="utf-8") as f:
    188             return cls(f.read(), ignore_metadata_errors=ignore_metadata_errors)
    189 

/usr/lib/python3.10/pathlib.py in open(self, mode, buffering, encoding, errors, newline)
   1117         if "b" not in mode:
   1118             encoding = io.text_encoding(encoding)
-> 1119         return self._accessor.open(self, mode, buffering, encoding, errors,
   1120                                    newline)
   1121 

IsADirectoryError: [Errno 21] Is a directory: 'Mottzerella/Llama-3-8B-Instruct-Finance-RAG'

Tried to push a saved model to Huggingface while logged into Google Colab. Tried different versions of the push including use_temp_dir=True, all with the same result. The model itself is listed in my Huggingface but none of the files are associated with it. Expected the model to be pushed, but got the above error.

Solution

  • according to https://github.com/huggingface/transformers/issues/28543 the issue stems from having a local path that has the same name as NEW_MODEL and a remote repository having the id NEW_MODEL as well with no model card. to fix this either :