docker teamcity youtrack upsource jetbrains-hub

How to connect youtrack, upsource and teamcity to hub all running as docker containers on the same machine

Since we are a small team we want to put JetBrains' Hub, Youtrack, Upsource and Teamcity as docker containers (all on the same machine for now). Docker is running on Photon OS 2.0 running on ESXi 6.7. Nginx in another container acts as a DNS proxy so all services are reachable with their own domain names on port 80 for now...

I got all 5 services running and and can access them in a browser. However connecting Youtrack, Upsource and Teamcity to Hub is a challenge. Youtrack, Upsource and Teamcity ask for the Hub URL to confirm it and ask for permission to access Hub.

The Problem:

Hub URL: http://hub.teamtools.mydomain.com -> the container can not access it under that address and verification fails with timeout
Hub URL: http://172.18.0.3:8080 -> the container can access Hub on the internal docker net and then shows a pop up which is trying to show a confirmation page by redirecting to Hub on that internal IP which of course fails in the browser (I tried to copy the URL from the popup into a new window and adjust it there as a hack but that does not work.)

Questions:

How can I link Youtrack, Upsource and Teamcity to the Hub? In order for the confimation process to work the docker containers need to be able to access each other with an external IP/domain name.
Does anything speak against having all four Teamtools on the same machine to get started and separate them later as demand grows?

Configuration so far:

The containers have been turned into services like so:

/etc/systemd/system/docker.nginx.servcie

[Unit]
Description=Nginx DNS proxy
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
Restart=always
ExecStartPre=/usr/bin/docker network create --subnet=172.18.0.0/16 dockerNet
ExecStartPre=-/usr/bin/docker exec %n stop
ExecStartPre=-/usr/bin/docker rm %n
ExecStartPre=/usr/bin/docker pull jwilder/nginx-proxy
ExecStart=/usr/bin/docker run --rm --name %n \
    -v /var/run/docker.sock:/tmp/docker.sock:ro \
    --net dockerNet --ip 172.18.0.2 \
    -p 80:80 \
    jwilder/nginx-proxy

[Install]
WantedBy=multi-user.target

/etc/systemd/system/docker.hub.service

[Unit]
Description=JetBrains Hub Service
After=docker.nginx-proxy.service
Requires=docker.nginx-proxy.service

[Service]
TimeoutStartSec=0
Restart=always
ExecStartPre=-/usr/bin/docker exec %n stop
ExecStartPre=-/usr/bin/docker rm %n
ExecStartPre=/usr/bin/docker pull jetbrains/hub:2018.2.9635
ExecStart=/usr/bin/docker run --rm --name %n \
    -v /opt/hub/data:/opt/hub/data \
    -v /opt/hub/conf:/opt/hub/conf \
    -v /opt/hub/logs:/opt/hub/logs \
    -v /opt/hub/backups:/opt/hub/backups \
    --net dockerNet --ip 172.18.0.3 \
    -p 8010:8080 \
    --expose 8080 \
    -e VIRTUAL_PORT=8080 \
    -e VIRTUAL_HOST=hub,teamtools.mydomain.com,hub.teamtools.mydomain.com \
    jetbrains/hub:2018.2.9635

[Install]
WantedBy=multi-user.target

... and so on. Since I'm, still trying things out, the ports are mapped on the host and exposed so nginx-proxy can pick them up. I also added static IPs to the containers hoping this would help with my problem.

Running those services results in:

CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS                    NAMES
7ba8ed89b832        jetbrains/teamcity-server         "/run-services.sh"       12 hours ago        Up 12 hours         0.0.0.0:8111->8111/tcp   docker.teamcity.service
5c819c48cbcc        jetbrains/upsource:2018.1.357     "/bin/bash /run.sh"      12 hours ago        Up 12 hours         0.0.0.0:8030->8080/tcp   docker.upsource.service
cf9dcd1b534c        jetbrains/youtrack:2018.2.42223   "/bin/bash /run.sh"      14 hours ago        Up 14 hours         0.0.0.0:8020->8080/tcp   docker.youtrack.service
de86c3e1f2e2        jetbrains/hub:2018.2.9635         "/bin/bash /run.sh"      14 hours ago        Up 14 hours         0.0.0.0:8010->8080/tcp   docker.hub.service
9df9cb44e485        jwilder/nginx-proxy               "/app/docker-entry..."   14 hours ago        Up 14 hours         0.0.0.0:80->80/tcp       docker.nginx-proxy.service

Additional Info:

I did consider this could be a firewall issue and this post seems to suggest the same thing:

https://forums.docker.com/t/access-docker-container-from-inside-of-the-container-via-external-url/33271

After some discussion with the provider of the virtual server it turned out, that conflicting firewall rules between plesk firewall and iptables caused this problem. After the conflict had been fixed by the provider the container could be accessed.

Firewall on Photon with rules auto added by docker:

Chain INPUT (policy DROP 2 packets, 203 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
  258 19408 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    6   360 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
  186 13066 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
  186 13066 DOCKER-ISOLATION  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
  103  7224 ACCEPT     all  --  *      br-83f08846fc2e  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    9   524 DOCKER     all  --  *      br-83f08846fc2e  0.0.0.0/0            0.0.0.0/0
   74  5318 ACCEPT     all  --  br-83f08846fc2e !br-83f08846fc2e  0.0.0.0/0            0.0.0.0/0
    1    52 ACCEPT     all  --  br-83f08846fc2e br-83f08846fc2e  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
  300 78566 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    8   472 ACCEPT     tcp  --  !br-83f08846fc2e br-83f08846fc2e  0.0.0.0/0            172.18.0.2           tcp dpt:80
    0     0 ACCEPT     tcp  --  !br-83f08846fc2e br-83f08846fc2e  0.0.0.0/0            172.18.0.3           tcp dpt:8080
    0     0 ACCEPT     tcp  --  !br-83f08846fc2e br-83f08846fc2e  0.0.0.0/0            172.18.0.5           tcp dpt:8080
    0     0 ACCEPT     tcp  --  !br-83f08846fc2e br-83f08846fc2e  0.0.0.0/0            172.18.0.4           tcp dpt:8080
    0     0 ACCEPT     tcp  --  !br-83f08846fc2e br-83f08846fc2e  0.0.0.0/0            172.18.0.6           tcp dpt:8111

Chain DOCKER-ISOLATION (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  br-83f08846fc2e docker0  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  docker0 br-83f08846fc2e  0.0.0.0/0            0.0.0.0/0
  186 13066 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
  186 13066 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Solution

It turned out to be a firewall issue.

I resolved it using this https://unrouted.io/2017/08/15/docker-firewall/ as a starting point.

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:FILTERS - [0:0]
:DOCKER-USER - [0:0]

-F INPUT
-F DOCKER-USER
-F FILTERS

-A INPUT -i lo -j ACCEPT
-A INPUT -p icmp --icmp-type any -j ACCEPT
-A INPUT -j FILTERS

-A DOCKER-USER -i eth0 -j FILTERS

-A FILTERS -m state --state ESTABLISHED,RELATED -j ACCEPT
# full access from my workstation
-A FILTERS -m state --state NEW -s 192.168.0.10/32
-A FILTERS -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A FILTERS -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A FILTERS -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A FILTERS -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT
-A FILTERS -j REJECT --reject-with icmp-host-prohibited

COMMIT