I am in Anaconda on Windows 10; I installed via:
conda install -c anaconda dask
conda install -c conda-forge modin
conda update conda
conda update anaconda
conda update dask
conda install -c conda-forge pandas=1.0.5 # this will also download modin 0.7.4-py_0 --> 0.8.0-py_0
So, consider the following example:
#!/usr/bin/env python3
import io
USEDASK=False
if not USEDASK:
import pandas as pd
else:
from dask.distributed import Client # SO:48067066
client = Client(processes=False) # create scheduler and worker automatically
#os.environ["MODIN_ENGINE"] = "dask" # Modin will use Dask
import modin.pandas as pd
my_csv_str = """Time[s], Channel 0
0.000000000000000, -0.736680805683136
0.000008000000000, -0.726485192775726
0.000016000000000, -0.721387386322021
0.000024000000000, -0.711191773414612
0.000032000000000, -0.700996160507202
0.000040000000000, -0.690800547599792
0.000048000000000, -0.670409321784973
0.000056000000000, -0.655115902423859
"""
my_csv_io = io.StringIO()
my_csv_io.write(my_csv_str)
my_csv_io.seek(0)
my_df = pd.read_csv(my_csv_io)
my_df.index = pd.to_timedelta(my_df.iloc[:,0], unit='s')
print(my_df)
When I have USEDASK=False, everything works as expected.
When I have USEDASK=True, I have the following failure:
python test\test.py
UserWarning: The Dask Engine for Modin is experimental.
UserWarning: Parameters provided defaulting to pandas implementation.
To request implementation, send an email to feature_requests@modin.org.
Traceback (most recent call last):
File "test\test.py", line 30, in <module>
my_df.index = pd.to_timedelta(my_df.iloc[:,0], unit='s')
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\tools\timedeltas.py", line 102, in to_timedelta
return _convert_listlike(arg, unit=unit, errors=errors)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\tools\timedeltas.py", line 140, in _convert_listlike
value = sequence_to_td64ns(arg, unit=unit, errors=errors, copy=False)[0]
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\arrays\timedeltas.py", line 961, in sequence_to_td64ns
data[mask] = iNaT
File "C:\ProgramData\Anaconda3\lib\site-packages\modin\pandas\series.py", line 337, in __setitem__
if key not in self.keys():
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\range.py", line 338, in __contains__
hash(key)
TypeError: unhashable type: 'Series'
Is there any way to get this code working using modin+dask?
It's not a good solution, but a workaround should be use:
my_df.index = pd.to_timedelta(my_df.iloc[:,0].values, unit='s')
this works for both USEDASK is True or False