dockernetworkingcontainerspodmanpodman-networking

Cannot access container through localhost or host's IP despite being properly bound but can access container through private IP address


I am trying to create a Podman container that runs my node-js webserver. Although for some reason, I can't access it through localhost:443, but I can access it through the containers private IP address, example (10.88.0.5:443)

You can see that when I curl the private IP:

curl https://10.88.0.5
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above

(it only fails because the SSL certificate is issued for my domain, but at least it can actually contact the website.)

curl https://localhost --connect-timeout 5
curl: (28) SSL connection timeout

connection times out.

Looking at

sudo netstat -tulp | grep https

you can see that the webserver is being properly bound to 443:

tcp       12      0 0.0.0.0:https           0.0.0.0:*               LISTEN      4991/conmon 

podman reports the same thing:

sudo podman port -l
443/tcp -> 0.0.0.0:443

Heres my Containerfile as well:

FROM node:22.2.0
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 443
CMD ["node", "."]

You can see that port 443 is exposed.

Heres the network configuration that the container is running on:

[
     {
          "name": "podman",
          "id": "2f259bab93aaaaa2542ba43ef33eb990d0999ee1b9924b557b7be53c0b7a1bb9",
          "driver": "bridge",
          "network_interface": "podman0",
          "created": "2024-07-10T12:43:23.144060349-04:00",
          "subnets": [
               {
                    "subnet": "10.88.0.0/16",
                    "gateway": "10.88.0.1"
               }
          ],
          "ipv6_enabled": false,
          "internal": false,
          "dns_enabled": false,
          "ipam_options": {
               "driver": "host-local"
          }
     }
]

and my routing table:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         mynetwork       0.0.0.0         UG    0      0        0 eno2
10.88.0.0       0.0.0.0         255.255.0.0     U     0      0        0 podman0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.100.0   0.0.0.0         255.255.255.0   U     0      0        0 eno2
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

I've tried disabling UFW. I've tried reinstalling Podman entirely. Neither have worked. Let me know if I'm doing something wrong or if you have anything you want me to try. I've been trying to fix this for like 4 days. If anyone can help that would be greatly appreciated.

FYI. I'm running the latest version of Debian.


Solution

  • Cockpit issues with podman.

    If you are using Cockpit and you're wondering why a container might not be properly accessible outside of the network, Cockpit might actually be the underlying cause. For whatever reason, even though bound properly, there seems to be an issue when initializing a container through Podman.

    Solution:

    The solution is pretty easy, just make sure that you create the container and run the container through Podman first. In my case, I had to use this command:

    sudo podman run -d -p 443:443/tcp --restart always --name webserver localhost/nodeserver
    

    Just remember that this is the syntax for using the publish flag (-p):

    -p=[[ip:][hostPort]:]containerPort[/protocol]
    

    as per documentation.

    End result:

    Now you have a container running that can actually be managed through Cockpit, as it appears that you only need to worry about initialization. So you can still stop, start, and use all the other management features that Cockpit provides as shown in the image below. I might decide to create a bug report on Cockpit's official Github. Image of container running inside Cockpit's management web interface.