pythonpipalpine-linuxgrpcio

`pip3 install grpcio` fails on Alpine Linux


I use Alpine Linux by Docker on my Mac (12.3.1) and try to run pip3 install grpcio but this command always fails.

I tried info here, but nothing worked until now.
Unable to install grpcio using pip install grpcio --> Upgrade to the latest setuptools https://github.com/grpc/grpc/issues/24390 --> Run export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 and export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

Step

  1. Build an image using this Dockerfile.

Dockerfile

FROM alpine:latest
COPY src /root/src # Please think this is empty. I don't use any files in this directory until now.
WORKDIR /root/src

RUN set -x \
    && apk update \
    && apk add build-base \
    && apk add python3 py3-pip python3-dev \
    && pip3 install --no-cache --upgrade pip setuptools \
    && pip3 install wheel
  1. Use docker run to get into the image.

  2. Run pip3 list to check what has been installed.

~/src # pip3 list
Package            Version
------------------ ---------
appdirs            1.4.4
CacheControl       0.12.10
certifi            2020.12.5
charset-normalizer 2.0.7
colorama           0.4.4
contextlib2        21.6.0
distlib            0.3.3
distro             1.6.0
html5lib           1.1
idna               3.3
lockfile           0.12.2
msgpack            1.0.2
ordered-set        4.0.2
packaging          20.9
pep517             0.12.0
pip                22.1.1
progress           1.6
pyparsing          2.4.7
requests           2.26.0
retrying           1.3.3
setuptools         62.3.2
six                1.16.0
toml               0.10.2
tomli              1.2.2
urllib3            1.26.7
webencodings       0.5.1
wheel              0.37.1
  1. Run pip3 install grpcio

This error message is too long to write in this question. Please check Google Docs for full messages. https://docs.google.com/document/d/1ATyMCA0vRAsxfDquByeWh7cE7InhPCG6bDsgtDEG2Ls/edit?usp=sharing https://docs.google.com/document/d/19erFzIcB2zCDbCklyeOGDVNUBTf6I8oW4B-sNWuO6Zk/edit?usp=sharing

Error messages (the last part)

(There are messages before this part. Please check Google Docs.)
      gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -g -fno-semantic-interposition -g -fno-semantic-interposition -g -fno-semantic-interposition -DTHREAD_STACK_SIZE=0x100000 -fPIC -D_WIN32_WINNT=1536 -DGRPC_XDS_USER_AGENT_NAME_SUFFIX=\"Python\" -DGRPC_XDS_USER_AGENT_VERSION_SUFFIX=\"1.46.3\" -DGPR_BACKWARDS_COMPATIBILITY_MODE=1 -DHAVE_CONFIG_H=1 -DGRPC_ENABLE_FORK_SUPPORT=1 "-DPyMODINIT_FUNC=extern \"C\" __attribute__((visibility (\"default\"))) PyObject*" -DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 -Isrc/python/grpcio -Iinclude -I. -Ithird_party/abseil-cpp -Ithird_party/address_sorting/include -Ithird_party/cares/cares/include -Ithird_party/cares -Ithird_party/cares/cares -Ithird_party/cares/config_linux -Ithird_party/re2 -Ithird_party/boringssl-with-bazel/src/include -Ithird_party/upb -Isrc/core/ext/upb-generated -Isrc/core/ext/upbdefs-generated -Ithird_party/xxhash -Ithird_party/zlib -I/usr/include/python3.9 -c third_party/cares/cares/src/lib/ares_process.c -o python_build/temp.linux-aarch64-cpython-39/third_party/cares/cares/src/lib/ares_process.o -std=c++11 -std=gnu99 -fvisibility=hidden -fno-wrapv -fno-exceptions -pthread
      cc1: warning: command-line option '-std=c++11' is valid for C++/ObjC++ but not for C
      creating None/tmp/tmp_x4urxfk
      gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -g -fno-semantic-interposition -g -fno-semantic-interposition -g -fno-semantic-interposition -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/include/python3.9 -c /tmp/tmp_x4urxfk/a.c -o None/tmp/tmp_x4urxfk/a.o
      Traceback (most recent call last):
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/unixccompiler.py", line 173, in _compile
          self.spawn(compiler_so + cc_args + [src, '-o', obj] +
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
          _classic_spawn(self, command)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py", line 917, in spawn
          spawn(cmd, dry_run=self.dry_run, **kwargs)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/spawn.py", line 68, in spawn
          raise DistutilsExecError(
      distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/src/python/grpcio/commands.py", line 280, in build_extensions
          build_ext.build_ext.build_extensions(self)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 448, in build_extensions
          self._build_extensions_serial()
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 473, in _build_extensions_serial
          self.build_extension(ext)
        File "/usr/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
          _build_ext.build_extension(self, ext)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 528, in build_extension
          objects = self.compiler.compile(sources,
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/src/python/grpcio/_parallel_compile_patch.py", line 58, in _parallel_compile
          multiprocessing.pool.ThreadPool(BUILD_EXT_COMPILER_JOBS).map(
        File "/usr/lib/python3.9/multiprocessing/pool.py", line 364, in map
          return self._map_async(func, iterable, mapstar, chunksize).get()
        File "/usr/lib/python3.9/multiprocessing/pool.py", line 771, in get
          raise self._value
        File "/usr/lib/python3.9/multiprocessing/pool.py", line 125, in worker
          result = (True, func(*args, **kwds))
        File "/usr/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
          return list(map(*args))
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/unixccompiler.py", line 176, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/setup.py", line 527, in <module>
          setuptools.setup(
        File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup
          return run_commands(dist)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
          dist.run_commands()
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "/usr/lib/python3.9/site-packages/setuptools/dist.py", line 1229, in run_command
          super().run_command(command)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/lib/python3.9/site-packages/setuptools/command/install.py", line 68, in run
          return orig.install.run(self)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 670, in run
          self.run_command('build')
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3.9/site-packages/setuptools/dist.py", line 1229, in run_command
          super().run_command(command)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 136, in run
          self.run_command(cmd_name)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3.9/site-packages/setuptools/dist.py", line 1229, in run_command
          super().run_command(command)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
          self.build_extensions()
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/src/python/grpcio/commands.py", line 284, in build_extensions
          raise CommandError(
      commands.CommandError: Failed `build_ext` step:
      Traceback (most recent call last):
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/unixccompiler.py", line 173, in _compile
          self.spawn(compiler_so + cc_args + [src, '-o', obj] +
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
          _classic_spawn(self, command)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py", line 917, in spawn
          spawn(cmd, dry_run=self.dry_run, **kwargs)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/spawn.py", line 68, in spawn
          raise DistutilsExecError(
      distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/src/python/grpcio/commands.py", line 280, in build_extensions
          build_ext.build_ext.build_extensions(self)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 448, in build_extensions
          self._build_extensions_serial()
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 473, in _build_extensions_serial
          self.build_extension(ext)
        File "/usr/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
          _build_ext.build_extension(self, ext)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 528, in build_extension
          objects = self.compiler.compile(sources,
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/src/python/grpcio/_parallel_compile_patch.py", line 58, in _parallel_compile
          multiprocessing.pool.ThreadPool(BUILD_EXT_COMPILER_JOBS).map(
        File "/usr/lib/python3.9/multiprocessing/pool.py", line 364, in map
          return self._map_async(func, iterable, mapstar, chunksize).get()
        File "/usr/lib/python3.9/multiprocessing/pool.py", line 771, in get
          raise self._value
        File "/usr/lib/python3.9/multiprocessing/pool.py", line 125, in worker
          result = (True, func(*args, **kwds))
        File "/usr/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
          return list(map(*args))
        File "/tmp/pip-install-dddnrveo/grpcio_87c868971a7943939c5252f5c860ad57/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/usr/lib/python3.9/site-packages/setuptools/_distutils/unixccompiler.py", line 176, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> grpcio

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Solution

  • The build environment for Alpine Linux is not installed by default.

    You need to install the header files - apk add linux-headers.

    This was in found in this github issue: grpcio can't be installed on alpine