dockeranacondadockerfilecondaconda-build

Conda fails to build, when inside docker container


I am trying to build a docker image. This is the full dockerfile:

FROM ubuntu

ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
ENV PATH /opt/conda/bin:$PATH

ENV TZ=Europe/Athens
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

RUN apt-get update --fix-missing && apt-get install -y wget bzip2 ca-certificates \
    libglib2.0-0 libxext6 libsm6 libxrender1 \
    git mercurial subversion

RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda2-4.5.11-Linux-x86_64.sh -O ~/miniconda.sh && \
    /bin/bash ~/miniconda.sh -b -p /opt/conda && \
    rm ~/miniconda.sh && \
    ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \
    echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
    echo "conda activate base" >> ~/.bashrc

RUN apt-get install -y curl grep sed dpkg && \
    TINI_VERSION=`curl https://github.com/krallin/tini/releases/latest | grep -o "/v.*\"" | sed 's:^..\(.*\).$:\1:'` && \
    curl -L "https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini_${TINI_VERSION}.deb" > tini.deb && \
    dpkg -i tini.deb && \
    rm tini.deb && \
    apt-get clean

ENTRYPOINT [ "/usr/bin/tini", "--" ]
CMD [ "/bin/bash" ]

#SECOND PART
RUN apt install -y libgl1-mesa-glx
RUN conda install conda-build
RUN apt-get install -y git
WORKDIR /
RUN git clone https://github.com/cadquery/cadquery.git
WORKDIR /cadquery
RUN conda env create -n cq -f environment.yml
RUN echo "source activate cq" > ~/.bashrc
ENV PATH /opt/conda/envs/cq/bin:$PATH
WORKDIR /testing

However, when it is time to conda build - more specifically on STEP 12, this line here:

RUN conda install conda-build

i get errors.

It seems to be installing the packages normally, then it fails.:

Proceed ([y]/n)? 
ruamel_yaml-0.15.100 | 268 KB    | ########## | 100% 
readline-8.1         | 464 KB    | ########## | 100% 
bzip2-1.0.8          | 105 KB    | ########## | 100% 
tzdata-2020f         | 123 KB    | ########## | 100% 
xz-5.2.5             | 438 KB    | ########## | 100% 
tk-8.6.10            | 3.2 MB    | ########## | 100% 
conda-build-3.21.4   | 585 KB    | ########## | 100% 
cffi-1.14.5          | 227 KB    | ########## | 100% 
ld_impl_linux-64-2.3 | 645 KB    | ########## | 100% 
urllib3-1.26.4       | 99 KB     | ########## | 100% 
pyyaml-5.4.1         | 180 KB    | ########## | 100% 
pip-21.1.1           | 2.0 MB    | ########## | 100% 
lz4-c-1.9.3          | 216 KB    | ########## | 100% 
beautifulsoup4-4.9.3 | 86 KB     | ########## | 100% 
python-3.9.5         | 22.7 MB   | ########## | 100% 
pkginfo-1.7.0        | 42 KB     | ########## | 100% 
tqdm-4.59.0          | 90 KB     | ########## | 100% 
setuptools-52.0.0    | 880 KB    | ########## | 100% 
python-libarchive-c- | 50 KB     | ########## | 100% 
cryptography-3.4.7   | 1.0 MB    | ########## | 100% 
icu-58.2             | 22.7 MB   | ########## | 100% 
pysocks-1.7.1        | 31 KB     | ########## | 100% 
libxml2-2.9.10       | 1.3 MB    | ########## | 100% 
certifi-2020.12.5    | 143 KB    | ########## | 100% 
openssl-1.1.1k       | 3.8 MB    | ########## | 100% 
libgcc-ng-9.1.0      | 8.1 MB    | ########## | 100% 
patchelf-0.12        | 92 KB     | ########## | 100% 
glob2-0.7            | 12 KB     | ########## | 100% 
idna-2.10            | 52 KB     | ########## | 100% 
liblief-0.10.1       | 2.0 MB    | ########## | 100% 
pycparser-2.20       | 94 KB     | ########## | 100% 
chardet-4.0.0        | 198 KB    | ########## | 100% 
py-lief-0.10.1       | 1.3 MB    | ########## | 100% 
markupsafe-2.0.1     | 22 KB     | ########## | 100% 
zlib-1.2.11          | 120 KB    | ########## | 100% 
wheel-0.36.2         | 31 KB     | ########## | 100% 
conda-4.10.1         | 3.1 MB    | ########## | 100% 
libffi-3.3           | 54 KB     | ########## | 100% 
yaml-0.2.5           | 87 KB     | ########## | 100% 
libarchive-3.4.2     | 1.6 MB    | ########## | 100% 
ca-certificates-2021 | 120 KB    | ########## | 100% 
conda-package-handli | 967 KB    | ########## | 100% 
filelock-3.0.12      | 10 KB     | ########## | 100% 
requests-2.25.1      | 51 KB     | ########## | 100% 
ncurses-6.2          | 1.1 MB    | ########## | 100% 
pytz-2021.1          | 244 KB    | ########## | 100% 
pycosat-0.6.3        | 108 KB    | ########## | 100% 
psutil-5.8.0         | 342 KB    | ########## | 100% 
sqlite-3.35.4        | 1.4 MB    | ########## | 100% 
zstd-1.4.9           | 809 KB    | ########## | 100% 
jinja2-3.0.0         | 99 KB     | ########## | 100% 
brotlipy-0.7.0       | 349 KB    | ########## | 100% 
ripgrep-12.1.1       | 1.5 MB    | ########## | 100% 
_libgcc_mutex-0.1    | 3 KB      | ########## | 100% 
six-1.15.0           | 13 KB     | ########## | 100% 
soupsieve-2.2.1      | 30 KB     | ########## | 100% 
pyopenssl-20.0.1     | 48 KB     | ########## | 100% 

Downloading and Extracting Packages

UnicodeDecodeError('ascii', '/info/test/tests/data/\xed\x94\x84\xeb\xa1\x9c\xea\xb7\xb8\xeb\x9e\xa8.zip.json', 22, 23, 'ordinal not in range(128)')


The command '/bin/sh -c conda install conda-build' returned a non-zero code: 1

Solution

  • Conda is Too Old

    I replicated this error with the continuumio/miniconda2:4.5.11 Docker image:

    $ docker run --rm -it continuumio/miniconda2:4.5.11 bash
    (base) root@a285050719ad:/# conda install -y conda-build
    
    # ... similar output as OP ...
    
    UnicodeDecodeError('ascii', '/info/test/tests/data/\xed\x94\x84\xeb\xa1\x9c\xea\xb7\xb8\xeb\x9e\xa8.zip.json', 22, 23, 'ordinal not in range(128)')
    

    Additionally, attempting to upgrade the conda package fails with some extra advice:

    (base) root@a285050719ad:/# conda update conda
    Solving environment: done
    
    EncodingError: A unicode encoding or decoding error has occurred.
    Python 2 is the interpreter under which conda is running in your base environment.
    Replacing your base environment with one having Python 3 may help resolve this issue.
    If you still have a need for Python 2 environments, consider using 'conda create'
    and 'conda activate'.  For example:
    
        $ conda create -n py2 python=2
        $ conda activate py2
    
    Error details: UnicodeDecodeError('ascii', '/info/test/tests/data/\xed\x94\x84\xeb\xa1\x9c\xea\xb7\xb8\xeb\x9e\xa8.zip.json', 22, 23, 'ordinal not in range(128)')
    

    That is, you really shouldn't be using these old Miniconda2 images because the conda is no longer compatible with the Anaconda Cloud repository.

    Use Newer Miniconda Installer

    A clean solution is to install a newer Miniconda (or Miniforge or Mambaforge). The latest ones all have Python 3 in the base. If for some reason one must have Python 2 in the base, which means you can't have the latest conda nor the latest conda-build, then it seems Miniconda up to 4.8.3 supported Python 2.

    If possible, use the latest version, as in Python 3. One can always create a Python 2 environment if needed - just better that it not be in the base. Suggested solution:

    RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && \
    ...
    

    Also, consider whether you can more simply start from an existing Docker image with Conda preinstalled.

    Update Conda in Place (Not Recommended)

    A dirty version would be still using the same 4.5.11 installer, but then upgrading immediately. In the Docker image I can get it to upgrade to 4.8 and keep Python 2.7, then conda-build will install at 3.18.11 (current as of May 2021 is 3.21.4).

    This could similarly be done in the Dockerfile with something like

    RUN conda install -y python=2.7 conda=4.8 && \
        conda clean -qafy && \
        conda install -y conda-build && \
        conda clean -qafy
    

    Note: I needed the first clean to get conda-build to install.