When using Google Colab, Python package 'datasets' just disappeared from virtualenv directory 'site-packages'

I'm using Google Colab and trying make a virtual environment to work.

My code is:

  from google.colab import drive
  drive.mount('/content/drive')

  !pip install virtualenv
  myenv_dir = '/content/drive/MyDrive/virtual_env/' 
  !virtualenv {myenv_dir}
  !chmod +x {myenv_dir}bin/pip;
  !chmod +x {myenv_dir}bin/activate;
  !source {myenv_dir}bin/activate; pip install accelerate==0.29.3 -U
  !source {myenv_dir}bin/activate; pip install datasets==2.19.1

  import sys
  packages_dir = myenv_dir + "lib/python3.10/site-packages/"
  sys.path.append(packages_dir)

  import accelerate
  import datasets

This code runs ok and at this moment I can import both accelerate and datasets packages. I look at the Google Drive file explorer and the subdirectories are there,both for accelerate and datasets.

Now I disconnect the notebook, reconnect it and run just the code below (enough to connect to the virtual environment and start using the packages without reinstalling them on Collab):

  drive.mount('/content/drive')
  myenv_dir = '/content/drive/MyDrive/virtual_env/' 
  !chmod +x {myenv_dir}bin/activate;
  !source {myenv_dir}bin/activate;
  import sys
  packages_dir = myenv_dir + "lib/python3.10/site-packages/"
  sys.path.append(packages_dir)
  import accelerate
  import datasets 

And here comes the weird part: import accelerate works fine. import datasets returns a

ModuleNotFoundError: No module named 'datasets'.

If look in the directory tree either in Colab's file explorer or in Google Drive the accelerate subdirectory is there but the datasets directory in gone. Just vanished.

I'm at a loss on what's happening here!

Try this:

!export PYTHONPATH='/content/drive/MyDrive/virtual_env/lib/python3.10/site-packages/'
print(sys.path) 
import datasets

Seems to be the order of sys.path is causing failure to find the library. check: https://www.devdungeon.com/content/python-import-syspath-and-pythonpath-tutorial