I'm using the transformers library in Google colab, and
When i am using TrainingArguments from transformers library i'm getting Import error with this code:
from transformers import TrainingArguments
training_args = TrainingArguments(
output_dir = "/content/our-model",
learning_rate=2e-5,
per_device_train_batch_size= 64,
per_device_eval_batch_size = 16,
num_train_epochs = 2,
weight_decay = 0.01,
evaluation_strategy = "epoch",
save_strategy = "epoch",
load_best_model_at_end = True,
push_to_hub = False
)
This is the error i'm getting:
<ipython-input-28-0518ea5ff407> in <cell line: 2>()
1 from transformers import TrainingArguments
----> 2 training_args = TrainingArguments(
3 output_dir = "/content/our-model",
4 learning_rate=2e-5,
5 per_device_train_batch_size= 64,
4 frames
/usr/local/lib/python3.10/dist-packages/transformers/training_args.py in _setup_devices(self)
1670 if not is_sagemaker_mp_enabled():
1671 if not is_accelerate_available(min_version="0.20.1"):
-> 1672 raise ImportError(
1673 "Using the `Trainer` with `PyTorch` requires `accelerate>=0.20.1`: Please run `pip install transformers[torch]` or `pip install accelerate -U`"
1674 )
ImportError: Using the `Trainer` with `PyTorch` requires `accelerate>=0.20.1`: Please run `pip install transformers[torch]` or `pip install accelerate -U
I already tried pip install for 0.20.1 version of accelerate and pip install transformers[torch] and both didn't worked.
If you're not particular about which transformers and accelerate version to tie to, then do this to use the most up-to-date version in Google Colab:
! pip install -U accelerate
! pip install -U transformers
Then the issue you are having with accelerate should auto-resolve itself.
Note:
Underspecifying pip install -U transformers instead of pip install transformers[pytorch] might be easier since that's what most of the users do and the developers of the library will make sure that the basic pip works with the common functions and class like TrainingArguments
Instead of specifying accelerate to the pip install accelerate>=0.20.1, if you have no particular need to fixed the version, automatically upgrading to the latest version might get you more stability when using the library, esp. with "hot"/"trending" libraries that are constantly changing (almost) daily.
If further debugging is necessary, i.e. if the above didn't work. To check your transformers and accelerate version, do this:
import accelerate
accelerate.__version__
Most probably you might have an ImportError at the first line if accelerate is not already installed when you installed transformers.
And then if the first line works and the 2nd line is not outputting a version >=0.20.1, then that is the cause of your issue.
The current versions to-date (July 2023) are:
import accelerate
import transformers
transformers.__version__, accelerate.__version__
[out]:
('4.30.1', '0.21.0')
Here's an example notebook with the model that you wish to use as per the comments in your question, https://colab.research.google.com/drive/1D79AjHMeE6HAZC-g2S83baTgsHtDUu5i?usp=sharing
If the error persist after the pip install ..., try restarting the runtime.
If you can't find the buttons to press to restart, try this in the cell Restart kernel in Google Colab then re-run the cells for import ...
import os
os._exit(00)