[SOLVED] CMake cannot find conda libboost in dockerfile

CMake cannot find conda libboost in dockerfile

Following is an abridged version of the Dockerfile I'm using

FROM pangeo/pangeo-notebook:2024.04.08 AS base
USER root
# Install dependencies
RUN apt-get update && apt-get install -y vim gfortran sqlite3 libsqlite3-dev ...
RUN mamba install -c conda-forge libboost -y #<--------------------------- libboost installation

FROM base AS prebuild
...

FROM prebuild AS app_clone
WORKDIR /app
RUN git clone ...

FROM app_clone AS app_build
ENV PATH=${PATH}:/usr/bin/mpich
ENV PATH=/usr/bin:${PATH} #<-------------- Env variable not showing in docker image 
ENV CC=/usr/bin/gcc #<------------- Env variable not showing in docker image
WORKDIR /app

RUN cmake -G Ninja -B cmake_build_serial -S . && \
    cmake --build cmake_build_serial --target all -- -j $(nproc)

I installed libboost using mamba since I want at least version 1.79.0. There are two issues happening

1. When I try to build the image using docker build --no-cache --platform linux/amd64 -t test-app ., it shows following error

0.788 -- The C compiler identification is GNU 11.4.0                                                                                                                                
1.204 -- The CXX compiler identification is GNU 11.4.0
1.238 -- Detecting C compiler ABI info
1.803 -- Detecting C compiler ABI info - done
1.822 -- Check for working C compiler: /usr/bin/gcc - skipped
1.823 -- Detecting C compile features
1.824 -- Detecting C compile features - done
1.828 -- Detecting CXX compiler ABI info
2.345 -- Detecting CXX compiler ABI info - done
2.365 -- Check for working CXX compiler: /usr/bin/g++ - skipped
2.365 -- Detecting CXX compile features
2.366 -- Detecting CXX compile features - done
2.384 CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
2.384   Could NOT find Boost (missing: Boost_INCLUDE_DIR) (Required is at least
2.384   version "1.79.0")
2.384 Call Stack (most recent call first):
2.384   /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
2.384   /usr/share/cmake-3.22/Modules/FindBoost.cmake:2360 (find_package_handle_standard_args)
2.384   CMakeLists.txt:168 (find_package)
2.384 
2.384 -- Configuring incomplete, errors occurred!
------

However,

if I comment the last line (cmake ...) in the dockerfile
run container docker run -it test-app /bin/bash
manually copy the cmake ... command

the compilation completes without any issue.

2. Also, when inside the container, the environment variables are not set i.e.

CC points to old value /srv/conda/envs/notebook/bin/x86_64-conda-linux-gnu-cc
PATH does not have /usr/bin as prefix

Can someone please point out what I'm missing here?

[SOLUTION] Thanks for all the comments here and on a related post which guided me to a solution posted here. What worked for me was to modify the Dockerfile to use/activate notebook environment during image building before cmake ... as follows

# Make RUN commands use the notebook environment
SHELL ["mamba", "run", "--no-capture-output", "-n", "notebook", "/bin/bash", "-c"]
RUN cmake -G Ninja -B cmake_build_serial -S . && \
    cmake --build cmake_build_serial --target all -- -j $(nproc)

Solution

The reason that cmake finds the library at runtime, but not whilst building the container is due to the way that the environment is set-up; the Docker image has some machinery configured to activate your mamba environment at runtime. But, during build time, the environment is not yet active, and so the dependencies are not visible.

The fix is to activate the mamba environment for the RUN step. However, it's really not advisable to mix conda and system packages like this. If you're needing to compile something, I'd suggest building boost yourself, or moving exclusively to conda.