How to replace load balancer Docker container with no downtime in a simple way

I'm running around 30 different sites on the same host, each managed using docker-compose. Some of them use Nginx, some use Apache. Then there's a reverse proxy container exposed to the internet which runs Nginx, has a bunch of vhosts and will redirect traffic to the right container. Those 30 different sites are just me trying out various ideas, and they don't have a lot of customers.

I know that using docker-compose is unwise for production use, because turning off and turning the container back on using the new version still takes time (<3 seconds). I know about docker swarm and K8S too, which I will use if any of those 30 different sites becomes big enough. By that point I'll probably restructure everything to achieve zero downtime deployment. But what I'm asking is different.

I want to still manage normal containers using docker compose, because it's just so convenient and because I only have 1 node, so running K8S would be overkill and wouldn't really have zero downtime deployment. However, I want to replace the reverse proxy container every now and then (may be 1-3 times a week) and I don't want to have any downtime with that single container because all those 30 sites will be affected. I can accept <3 seconds downtime for single sites, because most sites don't change at all, and the ones that do are experimental, but I'm starting to feel uncomfortable having <3 seconds downtime for the reverse proxy.

So is there a way for me to replace the reverse proxy container with no downtime? Something really simple and lightweight? Doesn't have to use docker compose, but should still use containers overall. I'm considering these:

• Use KIND (Kubernetes IN Docker), which runs on a single host and creates containers so that they appear as nodes. This takes 800MB-1GB RAM, so it's undesirable
• Keep the reverse proxy container up indefinitely, change the configs inside of it, restart Nginx (<1 seconds downtime), but this seriously defeats the purpose of containers and I can't just don't care about containers anymore, which is the initial reason I want to containerize the applications.

Are there any other options?

What I ended up doing is placing another load balancer (call it A) in front of 2 copies (call them B and C) of the current load balancer. Assuming A never have down time and never needs to be updated, I can update B (meanwhile traffic will go to A then C then app), then I can update C (meanwhile traffic will go to A then B then app). I know that Deployments and ReplicaSets in K8S do exactly this, but using K8S still can't be justified in my situation due to it being complex and heavy.

For anyone reading this, I still recommend using K8S, because K8S is legit and will help you manage your sites painlessly.