I am trying to install the Scrapy package (among others) for python using pip. I have tried doing the installation using python 3 and python 2, I have installed/upgraded the setuptools like so: $ pip3 install --upgrade setuptools, I have tried to use the --trusted-host option like so: $ pip3 install --trusted-host pypi.python.org Scrapy. But I always get the same error message when I run $ pip3 install Scrapy. The complete output is this:
Collecting Scrapy
Using cached Scrapy-1.3.2-py2.py3-none-any.whl
Collecting PyDispatcher>=2.0.5 (from Scrapy)
Using cached PyDispatcher-2.0.5.tar.gz
Collecting service-identity (from Scrapy)
Using cached service_identity-16.0.0-py2.py3-none-any.whl
Collecting pyOpenSSL (from Scrapy)
Using cached pyOpenSSL-16.2.0-py2.py3-none-any.whl
Collecting w3lib>=1.15.0 (from Scrapy)
Using cached w3lib-1.17.0-py2.py3-none-any.whl
Collecting parsel>=1.1 (from Scrapy)
Using cached parsel-1.1.0-py2.py3-none-any.whl
Collecting queuelib (from Scrapy)
Using cached queuelib-1.4.2-py2.py3-none-any.whl
Requirement already satisfied: six>=1.5.2 in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (from Scrapy)
Collecting Twisted>=13.1.0 (from Scrapy)
Using cached Twisted-17.1.0.tar.bz2
Complete output from command python setup.py egg_info:
Download error on https://pypi.python.org/simple/incremental/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749) -- Some packages may not be found!
Couldn't find index page for 'incremental' (maybe misspelled?)
Download error on https://pypi.python.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749) -- Some packages may not be found!
No local packages or working download links found for incremental>=16.10.1
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/gy/5xt04_452z791v1qjs1yzxkh0000gn/T/pip-build-nkv4jozy/Twisted/setup.py", line 21, in <module>
setuptools.setup(**_setup["getSetupArgs"]())
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/setuptools/dist.py", line 317, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/setuptools/dist.py", line 372, in fetch_build_eggs
replace_conflicting=True,
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 851, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1123, in best_match
return self.obtain(req, installer)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1135, in obtain
return installer(requirement)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/setuptools/dist.py", line 440, in fetch_build_egg
return cmd.easy_install(req)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 668, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('incremental>=16.10.1')
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/gy/5xt04_452z791v1qjs1yzxkh0000gn/T/pip-build-nkv4jozy/Twisted/
I am on a mac OS version 10.12.1 and am using python 3.6. Does anybody know a solution to this problem?
As stated here https://bugs.python.org/issue28150 in previous versions of python Apple supplied the OpenSSL packages but does not anymore.
Running the command pip install certifi and then pip install Scrapy fixed it for me