pythonsnappydatashadergeoviews

Datashader has snappy error


I was using python's datashader 0.5.0 package to plot population density information, generally following the tutorial https://www.continuum.io/blog/developer-blog/analyzing-and-visualizing-big-data-interactively-your-laptop-datashading-2010-us . I installed datashader using conda install -c bokeh datashader=0.5.0.

All was fine. Though perhaps unrelated, things seemed to break as soon as I installed the haloviews and geoviews packages. After installing these additional packages, I can no longer import datashader and my once working code no longer runs. When importing datashader, I get the following error: AttributeError: module 'snappy' has no attribute 'compress'

I am running on windows 10, anaconda python 3.5.3.

Perhaps I'm going down the wrong rabbit hole, but I thought perhaps it was the snappy package. I ran "conda install -c conda-forge snappy=1.1.4". conda list reveals that snappy is installed. Snappy does import. The snappy.compress object is not found. My issue seems related to the following SO post as I also had a fastparquet error when trying geoviews: error with snappy while importing fastparquet in python

When running import snappy, print(snappy.__filename__) gives the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-b8565733b383> in <module>()
----> 1 import snappy; print(snappy.__file__)

AttributeError: module 'snappy' has no attribute '__file__'

I also tried uninstalling through both conda and pip just in case. Still no joy.

Running "pip install python-snappy" results in a "failed building wheel for python-snappy" error preceded with " error: Microsoft Visual C++ 14.0 is required..." So I went and got the "Microsoft Visual C++ Redistributable for Visual Studio 2017" and ran it, but had no change.

Any thoughts on how to resolve this? For reference, the full error on datashader import is as follows:

--------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-3d7b1ff9e530> in <module>()
----> 1 import datashader

C:\Python\lib\site-packages\datashader\__init__.py in <module>()
      3 __version__ = '0.5.0'
      4 
----> 5 from .core import Canvas
      6 from .reductions import (count, any, sum, min, max, mean, std, var, count_cat,
      7                          summary)

C:\Python\lib\site-packages\datashader\core.py in <module>()
      3 import numpy as np
      4 from datashape.predicates import istabular
----> 5 from odo import discover
      6 from xarray import DataArray
      7 

C:\Python\lib\site-packages\odo\__init__.py in <module>()
     63     from .backends.url import URL
     64 with ignoring(ImportError):
---> 65     from .backends.dask import dask
     66 
     67 

C:\Python\lib\site-packages\odo\backends\dask.py in <module>()
      8 
      9 from dask.array.core import Array, from_array
---> 10 from dask.bag.core import Bag
     11 import dask.bag as db
     12 from dask.compatibility import long

C:\Python\lib\site-packages\dask\bag\__init__.py in <module>()
      1 from __future__ import absolute_import, division, print_function
      2 
----> 3 from .core import (Bag, Item, from_sequence, from_url, to_textfiles, concat,
      4                    from_delayed, map_partitions, bag_range as range,
      5                    bag_zip as zip, bag_map as map)

C:\Python\lib\site-packages\dask\bag\core.py in <module>()
     30 
     31 from ..base import Base, normalize_token, tokenize
---> 32 from ..bytes.core import write_bytes
     33 from ..compatibility import apply, urlopen
     34 from ..context import _globals, defer_to_globals

C:\Python\lib\site-packages\dask\bytes\__init__.py in <module>()
      2 
      3 from ..utils import ignoring
----> 4 from .core import read_bytes, open_files, open_text_files
      5 
      6 from . import local

C:\Python\lib\site-packages\dask\bytes\core.py in <module>()
      7 from warnings import warn
      8 
----> 9 from .compression import seekable_files, files as compress_files
     10 from .utils import (SeekableFile, read_block, infer_compression,
     11                     infer_storage_options, build_name_function)

C:\Python\lib\site-packages\dask\bytes\compression.py in <module>()
     30 with ignoring(ImportError):
     31     import snappy
---> 32     compress['snappy'] = snappy.compress
     33     decompress['snappy'] = snappy.decompress
     34 

AttributeError: module 'snappy' has no attribute 'compress'

Solution

  • It turns out that in adding packages, something messed up the snappy install. I followed this solution: How to install snappy C libraries on Windows 10 for use with python-snappy in Anaconda? It was a snappy error, not a datashader issue, but I'll leave the post in case anyone has the same series of issues.