dockervirtualizationqemukvmpodman

Podman in Podman, similar to Docker in Docker?


Is there a way to run Podman inside Podman, similar to the way you can run Docker inside Docker?

Here is a snippet of my Dockerfile which is strongly based on another question:

FROM debian:10.6

RUN apt update && apt upgrade -qqy && \
    apt install -qqy iptables bridge-utils \
                     qemu-kvm libvirt-daemon libvirt-clients virtinst libvirt-daemon-system \
                     cpu-checker kmod && \
    apt -qqy install curl sudo gnupg2 && \
    echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/Debian_10/ /" > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list && \
    curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/Debian_10/Release.key | sudo apt-key add - && \
    apt update && \
    apt -qqy install podman

Now trying some tests:

$ podman run -it my/test bash -c "podman --storage-driver=vfs info"
... (long output; this works fine)

$ podman run -it my/test bash -c "podman --storage-driver=vfs images"
ERRO[0000] unable to write system event: "write unixgram @000ec->/run/systemd/journal/socket: sendmsg: no such file or directory" 
REPOSITORY  TAG     IMAGE ID  CREATED  SIZE

$ podman run -it my/test bash -c "podman --storage-driver=vfs run docker.io/library/hello-world"
ERRO[0000] unable to write system event: "write unixgram @000ef->/run/systemd/journal/socket: sendmsg: no such file or directory" 
Trying to pull docker.io/library/hello-world...
Getting image source signatures
Copying blob 0e03bdcc26d7 done  
Copying config bf756fb1ae done  
Writing manifest to image destination
Storing signatures
ERRO[0003] unable to write pod event: "write unixgram @000ef->/run/systemd/journal/socket: sendmsg: no such file or directory" 
ERRO[0003] Error preparing container 66692b7ff496775499d405d538769a078f2794549955cf2409fcbcbf87f42e94: error creating network namespace for container 66692b7ff496775499d405d538769a078f2794549955cf2409fcbcbf87f42e94: mount --make-rshared /var/run/netns failed: "operation not permitted" 
Error: failed to mount shm tmpfs "/var/lib/containers/storage/vfs-containers/66692b7ff496775499d405d538769a078f2794549955cf2409fcbcbf87f42e94/userdata/shm": operation not permitted

I've also tried a suggestion from the other question, passing --cgroup-manager=cgroupfs, but without success:

$ podman run -it my/test bash -c "podman --storage-driver=vfs --cgroup-manager=cgroupfs run docker.io/library/hello-world"
Trying to pull docker.io/library/hello-world...
Getting image source signatures
Copying blob 0e03bdcc26d7 done  
Copying config bf756fb1ae done  
Writing manifest to image destination
Storing signatures
ERRO[0003] unable to write pod event: "write unixgram @000f3->/run/systemd/journal/socket: sendmsg: no such file or directory" 
ERRO[0003] Error preparing container c3fff4d8161903aaebd6f89f3b3c06b55038e11e07b6b561dc6576ca675747a3: error creating network namespace for container c3fff4d8161903aaebd6f89f3b3c06b55038e11e07b6b561dc6576ca675747a3: mount --make-rshared /var/run/netns failed: "operation not permitted" 
Error: failed to mount shm tmpfs "/var/lib/containers/storage/vfs-containers/c3fff4d8161903aaebd6f89f3b3c06b55038e11e07b6b561dc6576ca675747a3/userdata/shm": operation not permitted

Seems like some network configuration is needed. I found the project below which suggests that some tweaking on network configurations might be necessary, but I don't know what would be the context of that and whether it would apply here or not. https://github.com/joshkunz/qemu-docker

EDIT: I've just discovered /var/run/podman.sock, but also without success:

$ sudo podman run -it -v /run/podman/podman.sock:/run/podman/podman.sock my/test bash -c "podman --storage-driver=vfs --cgroup-manager=cgroupfs run docker.io/library/hello-world"
Trying to pull my/test...
  denied: requested access to the resource is denied
Trying to pull my:test...
  unauthorized: access to the requested resource is not authorized
Error: unable to pull my/text: 2 errors occurred:
        * Error initializing source docker://my/test: Error reading manifest latest in docker.io/my/test: errors:
denied: requested access to the resource is denied
unauthorized: authentication required

        * Error initializing source docker://quay.io/my/test:latest: Error reading manifest latest in quay.io/my/test: unauthorized: access to the requested resource is not authorized

Seems like root cannot see the images I've created under my user.

Any ideas? Thanks.


Solution

  • Assume we would like to run ls / in a docker.io/library/alpine container.

    Standard Podman

    podman run --rm docker.io/library/alpine ls /
    

    Podman in Podman

    Let's run ls / in a docker.io/library/alpine container, but this time we run podman in a quay.io/podman/stable container.

    Update June 2021

    A GitHub issue comment shows an example of how to run Podman in Podman as a non-root user both on the host and in the outer container. Slightly modified it would look like this:

    podman \
      run \
        --rm \
        --security-opt label=disable \
        --user podman \
        quay.io/podman/stable \
          podman \
            run \
              --rm \
              docker.io/library/alpine \
                ls / 
    

    Here is a full example:

    $ podman --version
    podman version 3.2.1
    $ cat /etc/fedora-release 
    Fedora release 34 (Thirty Four)
    $ uname -r
    5.12.11-300.fc34.x86_64
    $ podman \
      run \
        --rm \
        --security-opt label=disable \
        --user podman \
        quay.io/podman/stable \
          podman \
            run \
              --rm \
              docker.io/library/alpine \
                ls / 
    Trying to pull docker.io/library/alpine:latest...
    Getting image source signatures
    Copying blob sha256:5843afab387455b37944e709ee8c78d7520df80f8d01cf7f861aae63beeddb6b
    Copying config sha256:d4ff818577bc193b309b355b02ebc9220427090057b54a59e73b79bdfe139b83
    Writing manifest to image destination
    Storing signatures
    bin
    dev
    etc
    home
    lib
    media
    mnt
    opt
    proc
    root
    run
    sbin
    srv
    sys
    tmp
    usr
    var
    $ 
    

    To avoid repeatedly downloading the inner container image, create a volume

    podman volume create mystorage
    

    and add the command-line option -v mystorage:/home/podman/.local/share/containers:rw to the outer Podman command. In other words

    podman \
      run \
        -v mystorage:/home/podman/.local/share/containers:rw \
        --rm \
        --security-opt label=disable \
        --user podman \
        quay.io/podman/stable \
          podman \
            run \
              --rm \
              docker.io/library/alpine \
                ls / 
    

    Podman in Podman (outdated answer)

    (The old outdated answer from Dec 2020. I'll probably remove this when it's clear that the method described here is outdated)

    Let's run ls / in a docker.io/library/alpine container, but this time we run podman in a quay.io/podman/stable container.

    The command will look like this:

    podman \
      run \
        --privileged \
        --rm \
        --ulimit host \
        -v /dev/fuse:/dev/fuse:rw \
        -v ./mycontainers:/var/lib/containers:rw \
        quay.io/podman/stable \
          podman \
            run \
              --rm \
              --user 0 \
              docker.io/library/alpine ls 
    

    (The directory ./mycontainers is here used for container storage)

    Here is a full example

    $ podman --version
    podman version 2.1.1
    $ mkdir mycontainers
    $ podman run --privileged --rm --ulimit host -v /dev/fuse:/dev/fuse:rw -v ./mycontainers:/var/lib/containers:rw   quay.io/podman/stable podman run --rm --user 0 docker.io/library/alpine ls | head -5
    Trying to pull docker.io/library/alpine...
    Getting image source signatures
    Copying blob sha256:188c0c94c7c576fff0792aca7ec73d67a2f7f4cb3a6e53a84559337260b36964
    Copying config sha256:d6e46aa2470df1d32034c6707c8041158b652f38d2a9ae3d7ad7e7532d22ebe0
    Writing manifest to image destination
    Storing signatures
    bin
    dev
    etc
    home
    lib
    $ podman run --privileged --rm --ulimit host -v /dev/fuse:/dev/fuse:rw -v ./mycontainers:/var/lib/containers:rw  quay.io/podman/stable podman images
    REPOSITORY                TAG     IMAGE ID      CREATED     SIZE
    docker.io/library/alpine  latest  d6e46aa2470d  4 days ago  5.85 MB
    

    If you would leave out -v ./mycontainers:/var/lib/containers:rw you might see the slightly confusing error message

    Error: executable file `ls` not found in $PATH: No such file or directory: OCI runtime command not found error
    

    References: