I am quite new to programming and docker. I did come across such a problem, which I couldn't find an answer to.
I created my custom docker image with no problems and then I wanted to use this image as my base for further changes. So I created my Dockerfile, which looks like (see below) and when I use sudo docker build -t my-name . it executes with no problems, however when i run the image (sudo docker run -it my-name /bin/bash) I see no changes in the directory list (directory /root/new_files/ does not exist).
I seem to think this populated somehow from my first build on which my this image is based as I did these commands many times before with no problems.
Any advice, what could have caused Docker executing the program, but changes not seen in the image itself.
My Docker file:
FROM plu_build_1:latest
ENV BASEDIR=/root
WORKDIR /root
RUN cp -a $BASEDIR/TEMPLATE/ $BASEDIR/DEMO/
COPY DEMO/parameters.DEMO $BASEDIR/DEMO/
COPY DEMO/config.DEMO $BASEDIR/DEMO/
ENV PATH="${BASEDIR}/bin:${PATH}"
VOLUME ["/root/DEMO/LOG/"]
CMD ["bash"]
I want to note, that instead of cp -a I tried 'mv' command ... with no luck also seems that creating links with 'ln' does not work.
However if after build I enter image and run the same command inside running image it works fine, that means I can run 'cp -a $BASEDIR/TEMPLATE/ $BASEDIR/DEMO/' and it works.
So, --no-cache didn't help. Below is the base image Dockerfile:
base image Dockerfile:
FROM fedora:25
RUN yum -y update \
&& yum -y install file gcc gcc-gfortran gcc-c++ glibc.i686 libgcc.i686 libpng-devel jasper jasper-devel hostname m4 make perl \
tar tcsh time wget which zlib zlib-devel openssh-clients openssh-server net-tools \
netcdf-fortran libpng15 iproute-tc tcp_wrappers-libs sendmail procmail psmisc procps-ng mailx findutils ImageMagick \
perl-CPAN ncl netcdf libpng libjpeg-turbo which patch vim less bzip2 \
&& yum clean all
RUN yum -y install netcdf-openmpi-devel.x86_64 netcdf-fortran-openmpi-devel.x86_64 netcdf-fortran-openmpi.x86_64 hdf5-openmpi.x86_64 openmpi.x86_64 openmpi-devel.x86_64 \
&& yum clean all
COPY files.tgz /root
COPY files-bin.tgz /root
COPY rings.tgz /root
# extract all and link all files
RUN tar -xvzf files.tgz \
&& tar -xvzf files-bin.tgz \
&& tar -xvzf rings.tgz \
&& rm files*.tgz \
&& rm rings.tgz
WORKDIR /root/bin
COPY prog-cmake-linux.tar /root/bin
COPY files-cmake-linux.tar /root/bin
RUN tar xf prog-cmake-linux.tar \
&& tar xf files-cmake-linux.tar \
&& rm prog-cmake* \
&& rm files-cmake* \
&& rm -rdf /root/bin/test/ \
&& rm -rdf /root/bin/main/ \
&& rm -rdf /root/bin/*grid/ \
&& mkdir /wrf/netcdf_links \
&& ln -sf /usr/lib64/openmpi/lib /root/netcdf_links/lib \
&& ln -sf /usr/include/openmpi-x86_64 /root/netcdf_links/include
RUN (echo y;echo o conf prerequisites_policy follow;echo o conf commit) | cpan && cpan install Proc/Background.pm \
&& ln -s libnetcdff.so.6 /lib64/libnetcdff.so.5 \
&& ln -s libnetcdf.so.11 /lib64/libnetcdf.so.7
RUN echo export LDFLAGS="-lm" >> /etc/bashrc \
&& echo export NETCDF=/root/netcdf_links >> /etc/bashrc \
&& echo export JASPERINC=/usr/include/jasper/ >> /etc/bashrc \
&& echo export JASPERLIB=/usr/lib64/ >> /etc/bashrc \
&& echo export LD_LIBRARY_PATH="/usr/lib64/openmpi/lib" >> /etc/bashrc \
&& echo export PATH="/usr/lib64/openmpi/bin:$PATH" >> /etc/bashrc \
&& echo setenv LDFLAGS "-lm" >> /etc/csh.cshrc \
&& echo setenv NETCDF "/root/netcdf_links" >> /etc/csh.cshrc \
&& echo setenv JASPERINC "/usr/include/jasper/" >> /etc/csh.cshrc \
&& echo setenv JASPERLIB "/usr/lib64/" >> /etc/csh.cshrc \
&& echo setenv LD_LIBRARY_PATH "/usr/lib64/openmpi/lib" >> /etc/csh.cshrc \
&& echo setenv PATH "/usr/lib64/openmpi/bin:$PATH" >> /etc/csh.cshrc \
&& echo export BASEDIR=$BASEDIR >> /etc/bashrc \
&& echo export PATH+=:\$BASEDIR/bin >> /etc/bashrc
ENV LD_LIBRARY_PATH /usr/lib64/openmpi/lib
ENV PATH /usr/lib64/openmpi/bin:$PATH
# set up ssh configuration
COPY ssh_config /root/.ssh/config
RUN mkdir -p /root/.openmpi
COPY default-mca-params.conf /root/.openmpi/mca-params.conf
RUN mkdir -p /var/run/sshd \
&& ssh-keygen -A \
&& sed -i 's/#PermitRootLogin yes/PermitRootLogin yes/g' /etc/ssh/sshd_config \
&& sed -i 's/#RSAAuthentication yes/RSAAuthentication yes/g' /etc/ssh/sshd_config \
&& sed -i 's/#PubkeyAuthentication yes/PubkeyAuthentication yes/g' /etc/ssh/sshd_config \
&& ssh-keygen -f /root/.ssh/id_rsa -t rsa -N '' \
&& chmod 600 /root/.ssh/config \
&& chmod 700 /root/.ssh \
&& cp /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys
#
WORKDIR /root
VOLUME /root
The immediate cause of your problems is that, once you declare a directory a VOLUME in a Dockerfile, you can never make changes in that directory tree ever again. In particular, since your base image Dockerfile ends with
VOLUME /root
then a couple of steps later
FROM plu_build_1:latest # inherits that VOLUME
RUN cp -a /root/TEMPLATE/ /root/DEMO/
is a no-op because nothing in /root can ever be changed ever again.
The very short answer here is to never put VOLUME in a Dockerfile at all. It maybe makes sense for things like database servers that have a single directory tree that you almost always want to outlive a single container if you can, but that's an exception, and it has some confusing side effects (like this).
Looking at this Dockerfile more broadly, it looks like a full-blown development environment more than a self-contained reusable image. (It contains two network servers, two compiler stacks and a third interpreted language runtime, and an out-of-the-mainstream interactive shell; it goes out of its way to configure both shells' dotfiles, when many typical Docker paths run no shells at all; it contains an easily extracted ssh key that gives root permissions to something.) You might consider whether a stack built on Vagrant, a full-blown VM, and a more modular configuration management system like Ansible is a better match for what you're trying to build.