dockergoogle-compute-enginegoogle-container-os

how to set container ulimits in Container-Optimized OS


I need to set ulimits on the container. For example, docker run --ulimit memlock="-1:-1" <image>. However, I'm not sure how to do this when deploying a container-optimised VM on Compute Engine as it handles the startup of the container.

I'm able to deploy a VM with options like --privileged, -e for environment variables, and even an overriding CMD. How can I deploy a VM with ulimits set for the container?


Solution

  • I received an official reply:

    Unfortunately the Containers on Compute Engine feature does not currently support setting the ulimit options for containers.

    A workaround would be to set ulimit inside the container. For example: gcloud beta compute instances create-with-container INSTANCE --zone=ZONE --container-image=gcr.io/google-containers/busybox --container-privileged --container-command=sh --container-arg=-c --container-arg=ulimit\ -n\ 100000

    Unfortunately this method requires running the container as privileged.

    Best regards,...

    This reply gave me inspiration to do the following. Create a wrapper script that is referred to from your docker image's ENTRYPOINT. Within this wrapper script, set the ulimit(s) prior to starting the process(es) subjected to the ulimit(s).

    As a quick example:

    $HOME/example/wrapper.sh

    #! /bin/bash
    
    # set memlock to unlimited
    ulimit -l unlimited
    
    # start the elasticsearch node 
    # (found this from the base images dockerfile on github)
    /usr/local/bin/docker-entrypoint.sh eswrapper
    

    $HOME/example/Dockerfile

    FROM docker.elastic.co/elasticsearch/elasticsearch:6.3.2
    COPY wrapper.sh .
    RUN chmod 777 wrapper.sh
    ENTRYPOINT ./wrapper.sh
    

    local image build

    docker image build -t gcr.io/{GCLOUD_PROJECT_ID}/example:0.0.0 $HOME/example

    deploy to gcr.io

    docker push gcr.io/{GCLOUD_PROJECT_ID}/example:0.0.0

    create an instance via gcloud

    gcloud beta compute instances create-with-container example-instance-1 \
        --zone us-central1-a \
        --container-image=gcr.io/{GCLOUD_PROJECT_ID}/example:0.0.0 \
        --container-privileged \
        --service-account={DEFAULT_COMPUTE_ENGINE_SERVICE_ACC_ID}-compute@developer.gserviceaccount.com \
        --metadata=startup-script="echo 'vm.max_map_count=262144' > /etc/sysctl.conf; sysctl -p;"
    

    Note the following. The above startup script is only necessary for running a container of this image. The service account is necessary for pulling from your private google container registry. The --container-privileged argument is imperative as running the container with privileged is required to set ulimits within it.

    verifying ulimits are set for your process(es)

    On the vm HOST, ps -e and find the PID(s) of the process(es) that were executed within your wrapper script. In this case, find the PID whose command was java. For each PID, cat /proc/{PID}/limits. In this case, I only set memlock to unlimited. You can see that it is indeed set to unlimited.