pythonsnappyfastparquet

RuntimeError: Decompression 'SNAPPY' not available. Options: ['BROTLI', 'GZIP', 'UNCOMPRESSED'] (error happens only in .py and not in .ipython)


I got this error as in the title while trying to read parquet files using fastparquet with the following codes:

from fastparquet import ParquetFile
pf = ParquetFile(myfile.parquet)
df = pf.to_pandas()

I tried the solutions suggested from this post, but they did not work for me.

Afterwards, I also tried the following:
I uninstalled snappy from all my environments (as default python-snappy was also uninstalled together by conda). Then I installed python-snappy only in one environment - let's say env1 (as default snappy was installed together by conda). And I ran the same codes in .py and .ipython both in env1. Again .py did not work (same error as above) and .ipython worked...
Then I ran them in another environment (env2, where I did not installed python-snappy yet), and .ipython worked again!! And in this environment, .py did not work, which I can understand, since I did not install it yet. I am really confused what happened there and why .py won't work properly. I would appreciate any hints.

Details about packages and versions please see here


Solution

  • After trying around, I found a workaround: I downloaded the wheel file of python-snappy from the PyPI website and manually installed it via pip install --force-reinstall python_snappy-0.6.0-cp38-cp38-win_amd64.whl and this error disappeared. Hopefully, this can help others having the same issue.