We use Docker containers to deploy multiple small applications on our servers that are reachable on the public internet. Some of the services need to communicate to each other, but are deployed on different servers, due to different hardware requirements (the servers are on different network and different IP).
Q: What would be the best way to configure blocking of incoming requests to SERVER:PORT except for some allowed IPs and at the same time allow all outgoing connections of the Docker containers?
Two major things we played with and tried out to get them working:
Bound Docker port mappings to 127.0.0.1 and route every traffic through an nginx. This is really config heavy and some infrastructure components aren't possible to proxy via http(s), so we need to add them to nginx.conf stream-server block and therefore open a port on the server (that is accessible by everyone).
Use iptables to restrict access to the published ports. So something like this: iptables -A INPUT -I DOCKER-USER -p tcp -i eth0 -j DROP. But this also have 2 major downfalls. First it seems that it's quite hard to allow multiple IP adresses in such a construct and on the other hand this approach seems to block our docker outgoing connections (to the internet) as well. E. g.: After we activated it a ping google.com from within a docker container was rejected.
Not sure I get this. In term of design, what is available to the external world is in a DMZ or published through an API gateway.
Your docker swarm/kubernetes cluster shall not be accessible directly through the internet or only the API gateway or the application on the DMZ.
So quite likely your docker server shall not be accessible directly. And even if that is the case, if you don't explicitely export a port to the host/outside of the cluster, it stay restricted to the virtuals networks of docker to allow cross container communication.