ramazon-web-servicesdockerrcppamazon-linux

Docker image build not working for glmnet and Amazon Linux


I am creating a Dockerfile to run R workloads in Lambda based on a template from the lambdr package, which relies on an Amazon Linux base image and yum package manager. I need glmnet for one of the workloads I want to run. However, I can't get to install glmnet as it seems like there is some issue with the compilation.

This is the beginning of my Dockerfile:

FROM public.ecr.aws/lambda/provided:al2-x86_64

ENV R_VERSION=4.4.0

# Installs wget, git, tar, gcc and other development libraries
RUN yum -y install wget git tar gcc gcc-c++ libcurl-devel openssl-devel libxml2-devel

# Install EPEL
RUN yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm \
  # Install specific R version
  && wget https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm \
  && yum -y install R-${R_VERSION}-1-1.x86_64.rpm \
  && rm R-${R_VERSION}-1-1.x86_64.rpm
    
# Install AWS CLI
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \
    && unzip awscliv2.zip \
    && ./aws/install

# Accept AWS credentials as build arguments
ARG AWS_ACCESS_KEY_ID
ARG AWS_SECRET_ACCESS_KEY

# Set AWS credentials in the environment
ENV AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
ENV AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}


ENV PATH="${PATH}:/opt/R/${R_VERSION}/bin/"

# Creates a configuration file for R to set the C++ compiler.
RUN mkdir -p /root/.R && echo "CXX17 = g++" >> /root/.R/Makevars

RUN Rscript -e "install.packages('glmnet',  \
                                 INSTALL_opts =c('--no-docs', '--no-data',\
                                                  '--no-help', '--no-demo'),\
                                 repos = 'https://packagemanager.rstudio.com/all/__linux__/centos7/latest')"

I then call

docker build --build-arg AWS_ACCESS_KEY_ID={AWS_ACCESS_KEY_ID}--build-arg AWS_SECRET_ACCESS_KEY={AWS_SECRET_ACCESS_KEY} -t r-on-lambda .

Which outputs this at the end:

#11 107.7 /usr/bin/ld: RcppExports.o: relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: elnet_exp.o: relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: glmnet_init.o: relocation R_X86_64_32 against `.data' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: internal.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: pb_exp.o: relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: wls_exp.o: relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
#11 107.7 /usr/bin/ld: final link failed: Nonrepresentable section on output
#11 107.7 collect2: error: ld returned 1 exit status
#11 107.7 make: *** [glmnet.so] Error 1
#11 107.7 ERROR: compilation failed for package ‘glmnet’
#11 107.7 * removing ‘/opt/R/4.4.0/lib/R/library/glmnet’
#11 107.8 
#11 107.8 The downloaded source packages are in
#11 107.8   ‘/tmp/Rtmp7LKwru/downloaded_packages’
#11 107.8 Updating HTML index of packages in '.Library'
#11 107.8 Making 'packages.html' ... done
#11 107.8 Warning message:
#11 107.8 In install.packages("glmnet", INSTALL_opts = c("--no-docs", "--no-data",  :
#11 107.8   installation of package ‘glmnet’ had non-zero exit status
#11 DONE 107.9s

What is the reason for glmnet not being installed if I have gcc and gcc-c++ installed? In the log I have seen that all the dependencies (foreach, iterators, Rcpp, RcppEigen, shape) do get installed properly. How can I solve this?


Solution

  • I've discovered you can add the -fPIC flag into the CXX17 required by glmnet:

    Modifying this:

    RUN mkdir -p /root/.R && echo "CXX17 = g++" >> /root/.R/Makevars
    

    For that:

    RUN mkdir -p /root/.R && \
        echo "CXX17 = g++ -fPIC" >> /root/.R/Makevars && \
        echo "CXX17FLAGS = -O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function" >> /root/.R/Makevars