Pass environment variables from docker-compose to container at build stage

I am trying to create configuration for several environments with a single Dockerfile, several docker-compose files and several envoronment_variables files. I need it to use a different file with python dependencies for each environment.

Let's say, we create a web service in two environments: development and production. To achieve this, I create the following file structure:

docker-compose-dev.yml
docker-compose-prod.yml
envs/
  dev.env
  prod.env
web/
  Dockerfile
  requirements_dev.txt 
  requirements_prod.txt

The objective is to instantiate the proper name of a requirements_*.txt file in the Dockerfile during the container's build process. According to the documentation, I tried this naïve approach, which doesn't seem to work:

Define the name as an environment variable:
- envs/dev.env: REQUIREMENTS=requirements_dev.txt
- envs/prod.env: REQUIREMENTS=requirements_prod.txt

Use this environment variable in the Dockerfile:

FROM python:3.5
ENV PYTHONUNBUFFERED 1
ENV APP_ROOT /usr/src/app
...
COPY $REQUIREMENTS $APP_ROOT/
RUN pip install -r $APP_ROOT/$REQUIREMENTS

Import the corresponding definition in a docker-compose configuration:

version: '2'

services:
    web:
        build: ./web
        env_file:
            - envs/dev.env   # in docker-compose-dev.yml
            - envs/prod.env   # in docker-compose-prod.yml
        ...

When I run docker-compose up --build docker-compose-dev.yml, the $REQUIREMENTS variable is not defined at the build process.

In the docker-compose documentation there is a note on this:

Note: If your service specifies a build option, variables defined in environment files will not be automatically visible during the build. Use the args sub-option of build to define build-time environment variables.

But to use the ARG option in the Dockerfile, like ARG REQUIREMENTS, one needs a corresponding args option in docker-compose, like this:

services:
    web:
        build:
            context: ./web
            args:
                - REQUIREMENTS

Nevertheless, with this configuration my tests also failed, because, according to the documentation, the value of the REQUIREMENTS variable in Dockerfile is taken from

the environment where Compose is running

Therefore, it takes value from my shell, but not from envs/dev.env.

So my question is:

Is it possible to pass environment variable from env_file in docker-compose to Dockerfile at build time without setting variables locally in my shell?

Solution

Your docker-compose.yml should look like this:

version: '2'
services:
  web:
    build:
      context: ./web
      args:
        REQUIREMENTS: "requirements_dev.txt"

Your Dockerfile should define the build-argument using ARG like this:

FROM python:3.5
ENV PYTHONUNBUFFERED 1
ENV APP_ROOT /usr/src/app
ARG REQUIREMENTS
...
COPY $REQUIREMENTS $APP_ROOT/
RUN pip install -r $APP_ROOT/$REQUIREMENTS

I validated this and created a minified functional demo at Github:

https://github.com/jannikweichert/stackoverflow-41747843