huggingface-transformersjupyterhubhuggingface

Unable to use huggingface-cli in Jupyterhub

I created my model using HuggingFace Transformers. When I try to push to HuggingFace Hub, it shows the following error when I execute trainer.push_to_hub():

OSError: Failed to start custom transfer command "huggingface-cli" remote: exec: 
"huggingface-cli": executable file not found in $PATH
error: failed to push some refs to 'https://my_hugging_face_url_with_login_token'

The command can be used in the Terminal opened in Jupyterhub without a problem. How to resolve this PATH issue in Jupyterhub?

Solution

  • huggingface-hub in fact has a Python implementation:

    To login to the HuggingFace, we can use the following codes (if you don't mind to display the token in plain text in your code):

    import huggingface_hub
huggingface_hub.login('hf_TOKEN_HERE')

    Executing !huggingface-cli login in Notebook is not always working, as the token input box might not appear in some Notebook versions and requires user interaction.