nginxsslload-balancingconsulnginx-upstreams

Nginx Load Balancing HTTPs cluster


I watn to use Nginx as load balancer for Consul cluster. The Consul cluster is reachable only with TLS.

Here I've tried to reverse proxy a single Consul server to check if the TLS certificates are working

server {
    listen 80;
    listen [::]:80;
    
    location  /consul/ {

        resolver 127.0.0.1;

        proxy_pass https://core-consul-server-1-dev.company.io:8500;

        sub_filter_types text/css application/javascript;
        sub_filter_once off;
        sub_filter /v1/ /consul_v1/;

        proxy_ssl_certificate      /etc/nginx/certs/agent.crt;
        proxy_ssl_certificate_key  /etc/nginx/certs/agent.key;
        proxy_ssl_trusted_certificate  /etc/nginx/certs/ca.crt;
        proxy_ssl_verify        on;
        proxy_ssl_verify_depth  4;      

    }
}

this configuration working fine and I can call it with

curl http://core-proxy-server-1-dev.company.io/consul/consul_v1/agent/members

Now I've tried to do an upstream like this:

upstream consul {
    server core-consul-server-1-dev.company.io:8500;
    server core-consul-server-2-dev.company.io:8500;
}

server {

    listen 80;
    listen [::]:80;
  
    
    location  /consul/ {

        resolver 127.0.0.1;

        proxy_pass https://consul;
        sub_filter_types text/css application/javascript;
        sub_filter_once off;
        sub_filter /v1/ /consul_v1/;
        
        proxy_ssl_certificate      /etc/nginx/certs/agent.crt;
        proxy_ssl_certificate_key  /etc/nginx/certs/agent.key;
        proxy_ssl_trusted_certificate  /etc/nginx/certs/ca.crt;
        proxy_ssl_verify        on;
        proxy_ssl_verify_depth  4;      

    } 
}

when calling the same curl command as before, I get the following error:

2021/04/20 08:38:59 [debug] 3364#3364: *1 X509_check_host(): no match
2021/04/20 08:38:59 [error] 3364#3364: *1 upstream SSL certificate does not match "consul" while SSL handshaking to upstream, client: 10.10.xx.xxx, server: , request: "GET /consul/consul_v1/agent/members HTTP/1.1", upstream: "https://10.10.yy.yyy:8500/consul/consul_v1/agent/members", host: "core-proxy-server-1-dev.company.io"

Then I've tried like this:

upstream consul_1 {
    server core-consul-server-1-dev.company.io:8500;
}

upstream consul_2 {
    server core-consul-server-2-dev.company.io:8500;
}

map $http_host $backend {
    core-consul-server-1-dev.company.io       consul_1;
    core-consul-server-2-dev.company.io       consul_2;

}

server {

    listen 80;
    listen [::]:80;
  
    
    location  /consul/ {

        resolver 127.0.0.1;

        proxy_pass https://$backend;
        sub_filter_types text/css application/javascript;
        sub_filter_once off;
        sub_filter /v1/ /consul_v1/;
        
        proxy_ssl_certificate      /etc/nginx/certs/agent.crt;
        proxy_ssl_certificate_key  /etc/nginx/certs/agent.key;
        proxy_ssl_trusted_certificate  /etc/nginx/certs/ca.crt;
        proxy_ssl_verify        on;
        proxy_ssl_verify_depth  4;      

    }

}

but also no luck;

2021/04/20 08:45:05 [error] 3588#3588: *1 invalid URL prefix in "https://", client: 10.10.xx.xxx, server: , request: "GET /consul/consul_v1/agent/members HTTP/1.1", host: "core-proxy-server-1-dev.company.io"

any ideas? can someone please help me with one?


Solution

  • I figured it out.

    in this variation:

    upstream consul {
        server core-consul-server-1-dev.company.io:8500;
        server core-consul-server-2-dev.company.io:8500;
    }
    
    server {
    
        listen 80;
        listen [::]:80;
      
        
        location  /consul/ {
    
            resolver 127.0.0.1;
    
            proxy_pass https://consul;
            sub_filter_types text/css application/javascript;
            sub_filter_once off;
            sub_filter /v1/ /consul_v1/;
            
            proxy_ssl_certificate      /etc/nginx/certs/agent.crt;
            proxy_ssl_certificate_key  /etc/nginx/certs/agent.key;
            proxy_ssl_trusted_certificate  /etc/nginx/certs/ca.crt;
            proxy_ssl_verify        on;
            proxy_ssl_verify_depth  4;      
    
        } 
    }
    

    the upstream name consul should also match the alt_names I defined in my certificate. so changing the configuration to the one below did the trick:

    upstream core-consul-server-1-dev.company.io{
        server core-consul-server-1-dev.company.io:8500;
        server core-consul-server-2-dev.company.io:8500;
    }
    
    server {
    
        listen 80;
        listen [::]:80;
      
        
        location  /consul/ {
    
            resolver 127.0.0.1;
    
            proxy_pass https://core-consul-server-1-dev.company.io;
            sub_filter_types text/css application/javascript;
            sub_filter_once off;
            sub_filter /v1/ /consul_v1/;
            
            proxy_ssl_certificate      /etc/nginx/certs/agent.crt;
            proxy_ssl_certificate_key  /etc/nginx/certs/agent.key;
            proxy_ssl_trusted_certificate  /etc/nginx/certs/ca.crt;
            proxy_ssl_verify        on;
            proxy_ssl_verify_depth  4;      
    
        } 
    }
    

    I should just later add a general name in alt_names so I can reference the stream as

    core-consul-server-dev.company.io