SSL_do_handshake() failed (SSL: error:0A00010B:SSL routines::wrong version number) while SSL handshaking to upstream nginx-proxy
- I am trying to setup my docker compose node.js express application that uses redis postgres and nginx-proxy to manage certificates
- I am using the test or staging version of letsencrypt currently on my subdomains and
It keeps giving me this error
nginx-proxy | 2023/05/12 08:21:53 [error] 39#39: *8 SSL_do_handshake() failed (SSL: error:0A00010B:SSL routines::wrong version number) while SSL handshaking to upstream, client: 183.87.21.243, server: www.api.coinhexa.com, request: "GET /admin HTTP/2.0", upstream: "https://172.18.0.7:21347/admin", host: "www.api.coinhexa.com"
nginx-proxy | www.api.coinhexa.com 183.87.21.243 - - [12/May/2023:08:21:53 +0000] "GET /admin HTTP/2.0" 502 157 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/112.0" "172.18.0.7:21347"
nginx-proxy | 2023/05/12 08:21:54 [error] 39#39: *8 SSL_do_handshake() failed (SSL: error:0A00010B:SSL routines::wrong version number) while SSL handshaking to upstream, client: 183.87.21.243, server: www.api.coinhexa.com, request: "GET /favicon.ico HTTP/2.0", upstream: "https://172.18.0.7:21347/favicon.ico", host: "www.api.coinhexa.com", referrer: "https://www.api.coinhexa.com/admin"
nginx-proxy | www.api.coinhexa.com 183.87.21.243 - - [12/May/2023:08:21:54 +0000] "GET /favicon.ico HTTP/2.0" 502 157 "https://www.api.coinhexa.com/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/112.0" "172.18.0.7:21347"
Visually I see this bad gateway error. I double checked my IP address on EC2 and the domain provider to see if everything is pointed in the right direction and it seems fine.
Just to verify that is not an issue with my node.js app, I tried the IP directly and it works just fine without SSL
I ran nginx -T inside my docker container to get the loaded nginx configuration
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
# configuration file /etc/nginx/mime.types:
types {
text/html html htm shtml;
text/css css;
text/xml xml;
image/gif gif;
image/jpeg jpeg jpg;
application/javascript js;
application/atom+xml atom;
application/rss+xml rss;
text/mathml mml;
text/plain txt;
text/vnd.sun.j2me.app-descriptor jad;
text/vnd.wap.wml wml;
text/x-component htc;
image/avif avif;
image/png png;
image/svg+xml svg svgz;
image/tiff tif tiff;
image/vnd.wap.wbmp wbmp;
image/webp webp;
image/x-icon ico;
image/x-jng jng;
image/x-ms-bmp bmp;
font/woff woff;
font/woff2 woff2;
application/java-archive jar war ear;
application/json json;
application/mac-binhex40 hqx;
application/msword doc;
application/pdf pdf;
application/postscript ps eps ai;
application/rtf rtf;
application/vnd.apple.mpegurl m3u8;
application/vnd.google-earth.kml+xml kml;
application/vnd.google-earth.kmz kmz;
application/vnd.ms-excel xls;
application/vnd.ms-fontobject eot;
application/vnd.ms-powerpoint ppt;
application/vnd.oasis.opendocument.graphics odg;
application/vnd.oasis.opendocument.presentation odp;
application/vnd.oasis.opendocument.spreadsheet ods;
application/vnd.oasis.opendocument.text odt;
application/vnd.openxmlformats-officedocument.presentationml.presentation
pptx;
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
xlsx;
application/vnd.openxmlformats-officedocument.wordprocessingml.document
docx;
application/vnd.wap.wmlc wmlc;
application/wasm wasm;
application/x-7z-compressed 7z;
application/x-cocoa cco;
application/x-java-archive-diff jardiff;
application/x-java-jnlp-file jnlp;
application/x-makeself run;
application/x-perl pl pm;
application/x-pilot prc pdb;
application/x-rar-compressed rar;
application/x-redhat-package-manager rpm;
application/x-sea sea;
application/x-shockwave-flash swf;
application/x-stuffit sit;
application/x-tcl tcl tk;
application/x-x509-ca-cert der pem crt;
application/x-xpinstall xpi;
application/xhtml+xml xhtml;
application/xspf+xml xspf;
application/zip zip;
application/octet-stream bin exe dll;
application/octet-stream deb;
application/octet-stream dmg;
application/octet-stream iso img;
application/octet-stream msi msp msm;
audio/midi mid midi kar;
audio/mpeg mp3;
audio/ogg ogg;
audio/x-m4a m4a;
audio/x-realaudio ra;
video/3gpp 3gpp 3gp;
video/mp2t ts;
video/mp4 mp4;
video/mpeg mpeg mpg;
video/quicktime mov;
video/webm webm;
video/x-flv flv;
video/x-m4v m4v;
video/x-mng mng;
video/x-ms-asf asx asf;
video/x-ms-wmv wmv;
video/x-msvideo avi;
}
# configuration file /etc/nginx/conf.d/default.conf:
# nginx-proxy
# Networks available to the container running docker-gen (which are assumed to
# match the networks available to the container running nginx):
# coinhexa_prod_network
# If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
# scheme used to connect to this server
map $http_x_forwarded_proto $proxy_x_forwarded_proto {
default $http_x_forwarded_proto;
'' $scheme;
}
map $http_x_forwarded_host $proxy_x_forwarded_host {
default $http_x_forwarded_host;
'' $http_host;
}
# If we receive X-Forwarded-Port, pass it through; otherwise, pass along the
# server port the client connected to
map $http_x_forwarded_port $proxy_x_forwarded_port {
default $http_x_forwarded_port;
'' $server_port;
}
# If the request from the downstream client has an "Upgrade:" header (set to any
# non-empty value), pass "Connection: upgrade" to the upstream (backend) server.
# Otherwise, the value for the "Connection" header depends on whether the user
# has enabled keepalive to the upstream server.
map $http_upgrade $proxy_connection {
default upgrade;
'' $proxy_connection_noupgrade;
}
map $upstream_keepalive $proxy_connection_noupgrade {
# Preserve nginx's default behavior (send "Connection: close").
default close;
# Use an empty string to cancel nginx's default behavior.
true '';
}
# Abuse the map directive (see <https://stackoverflow.com/q/14433309>) to ensure
# that $upstream_keepalive is always defined. This is necessary because:
# - The $proxy_connection variable is indirectly derived from
# $upstream_keepalive, so $upstream_keepalive must be defined whenever
# $proxy_connection is resolved.
# - The $proxy_connection variable is used in a proxy_set_header directive in
# the http block, so it is always fully resolved for every request -- even
# those where proxy_pass is not used (e.g., unknown virtual host).
map "" $upstream_keepalive {
# The value here should not matter because it should always be overridden in
# a location block (see the "location" template) for all requests where the
# value actually matters.
default false;
}
# Apply fix for very long server names
server_names_hash_bucket_size 128;
# Default dhparam
# Set appropriate X-Forwarded-Ssl header based on $proxy_x_forwarded_proto
map $proxy_x_forwarded_proto $proxy_x_forwarded_ssl {
default off;
https on;
}
gzip_types text/plain text/css application/javascript application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
log_format vhost '$host $remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$upstream_addr"';
access_log off;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384';
ssl_prefer_server_ciphers off;
error_log /dev/stderr;
# HTTP 1.1 support
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $proxy_x_forwarded_host;
proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
proxy_set_header X-Forwarded-Ssl $proxy_x_forwarded_ssl;
proxy_set_header X-Forwarded-Port $proxy_x_forwarded_port;
proxy_set_header X-Original-URI $request_uri;
# Mitigate httpoxy attack (see README for details)
proxy_set_header Proxy "";
server {
server_name _; # This is just an invalid value which will never trigger on a real hostname.
server_tokens off;
listen 80;
listen 443 ssl http2;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;
access_log /var/log/nginx/access.log vhost;
ssl_certificate /etc/nginx/certs/default.crt;
ssl_certificate_key /etc/nginx/certs/default.key;
return 503;
}
# api.coinhexa.com/
upstream api.coinhexa.com {
# Container: api_server_prod
# networks:
# coinhexa_prod_network (reachable)
# IP address: 172.18.0.7
# exposed ports: 21347/tcp
# default port: 21347
# using port: 21347
# /!\ WARNING: Virtual port published on host. Clients
# might be able to bypass nginx-proxy and
# access the container's server directly.
server 172.18.0.7:21347;
}
server {
server_name api.coinhexa.com;
listen 80 ;
access_log /var/log/nginx/access.log vhost;
# Do not HTTPS redirect Let's Encrypt ACME challenge
location ^~ /.well-known/acme-challenge/ {
auth_basic off;
auth_request off;
allow all;
root /usr/share/nginx/html;
try_files $uri =404;
break;
}
location / {
return 301 https://$host$request_uri;
}
}
server {
server_name api.coinhexa.com;
access_log /var/log/nginx/access.log vhost;
listen 443 ssl http2 ;
ssl_session_timeout 5m;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;
ssl_certificate /etc/nginx/certs/api.coinhexa.com.crt;
ssl_certificate_key /etc/nginx/certs/api.coinhexa.com.key;
ssl_dhparam /etc/nginx/certs/api.coinhexa.com.dhparam.pem;
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/nginx/certs/api.coinhexa.com.chain.pem;
set $sts_header "";
if ($https) {
set $sts_header "max-age=31536000";
}
add_header Strict-Transport-Security $sts_header always;
include /etc/nginx/vhost.d/default;
location / {
proxy_pass https://api.coinhexa.com;
set $upstream_keepalive false;
}
}
# www.api.coinhexa.com/
upstream www.api.coinhexa.com {
# Container: api_server_prod
# networks:
# coinhexa_prod_network (reachable)
# IP address: 172.18.0.7
# exposed ports: 21347/tcp
# default port: 21347
# using port: 21347
# /!\ WARNING: Virtual port published on host. Clients
# might be able to bypass nginx-proxy and
# access the container's server directly.
server 172.18.0.7:21347;
}
server {
server_name www.api.coinhexa.com;
listen 80 ;
access_log /var/log/nginx/access.log vhost;
# Do not HTTPS redirect Let's Encrypt ACME challenge
location ^~ /.well-known/acme-challenge/ {
auth_basic off;
auth_request off;
allow all;
root /usr/share/nginx/html;
try_files $uri =404;
break;
}
location / {
return 301 https://$host$request_uri;
}
}
server {
server_name www.api.coinhexa.com;
access_log /var/log/nginx/access.log vhost;
listen 443 ssl http2 ;
ssl_session_timeout 5m;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;
ssl_certificate /etc/nginx/certs/www.api.coinhexa.com.crt;
ssl_certificate_key /etc/nginx/certs/www.api.coinhexa.com.key;
ssl_dhparam /etc/nginx/certs/www.api.coinhexa.com.dhparam.pem;
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/nginx/certs/www.api.coinhexa.com.chain.pem;
set $sts_header "";
if ($https) {
set $sts_header "max-age=31536000";
}
add_header Strict-Transport-Security $sts_header always;
include /etc/nginx/vhost.d/default;
location / {
proxy_pass https://www.api.coinhexa.com;
set $upstream_keepalive false;
}
}
# configuration file /etc/nginx/vhost.d/default:
## Start of configuration add by letsencrypt container
location ^~ /.well-known/acme-challenge/ {
auth_basic off;
auth_request off;
allow all;
root /usr/share/nginx/html;
try_files $uri =404;
break;
}
## End of configuration add by letsencrypt container
Here is my docker-compose file
version: '3.9' # optional since v1.27.0
name: coinhexa_api_prod
services:
api_server_prod:
build:
context: ../../
dockerfile: ./docker/production/api_server/Dockerfile
container_name: api_server_prod
depends_on:
cache_server_prod:
condition: service_healthy
database_server_prod:
condition: service_healthy
environment:
- ACME_OCSP=true
- DEBUG=1
- DEFAULT_EMAIL=something@gmail.com
- LETSENCRYPT_EMAIL=something@gmail.com
- LETSENCRYPT_HOST=api.coinhexa.com,www.api.coinhexa.com
- LETSENCRYPT_TEST=true
- VIRTUAL_HOST=api.coinhexa.com,www.api.coinhexa.com
- VIRTUAL_PORT=21347
- VIRTUAL_PROTO=https
env_file:
- .env
image: api_server_prod_image
networks:
- network
restart: 'always'
ports:
- '21347:21347'
cache_server_prod:
build:
context: ../..
dockerfile: ./docker/production/cache_server/Dockerfile
container_name: cache_server_prod
env_file:
- .env
healthcheck:
test:
['CMD-SHELL', 'redis-cli -p 23431 -a $REDIS_PASSWORD ping | grep PONG']
interval: 5s
timeout: 5s
retries: 3
start_period: 10s
image: cache_server_prod_image
networks:
- network
ports:
- '23431:23431'
restart: 'always'
volumes:
- cache_volume:/data
database_server_prod:
build:
context: ../..
dockerfile: ./docker/production/database_server/Dockerfile
container_name: database_server_prod
env_file:
- .env
healthcheck:
test:
[
'CMD-SHELL',
"pg_isready -d 'host=database_server_prod user=coinhexa_api_user port=26189 dbname=coinhexa_api_db'",
]
interval: 5s
timeout: 5s
retries: 3
start_period: 10s
image: database_server_prod_image
networks:
- network
ports:
- '26189:26189'
restart: 'always'
volumes:
- database_volume:/var/lib/postgresql/data
nginx-proxy:
image: nginx:alpine
container_name: nginx-proxy
ports:
- '80:80'
- '443:443'
volumes:
- conf:/etc/nginx/conf.d
- vhost:/etc/nginx/vhost.d
- html:/usr/share/nginx/html
- certs:/etc/nginx/certs:ro
networks:
- network
docker-gen:
image: jwilder/docker-gen
container_name: nginx-proxy-gen
command: -notify-sighup nginx-proxy -watch /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf
volumes_from:
- nginx-proxy
volumes:
- /home/ec2-user/api/docker/production/web_server/nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl:ro
- /var/run/docker.sock:/tmp/docker.sock:ro
labels:
- 'com.github.jrcs.letsencrypt_nginx_proxy_companion.docker_gen'
networks:
- network
acme-companion:
image: nginxproxy/acme-companion
container_name: nginx-proxy-acme
depends_on:
- docker-gen
- nginx-proxy
volumes_from:
- nginx-proxy
volumes:
- certs:/etc/nginx/certs:rw
- acme:/etc/acme.sh
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
- network
networks:
network:
driver: bridge
volumes:
cache_volume:
driver: local
database_volume:
driver: local
conf:
driver: local
vhost:
driver: local
html:
driver: local
certs:
driver: local
acme:
driver: local
Can someone kindly tell me how to fix this error?
Solution
This is likely caused by this block :
location / {
proxy_pass https://www.api.coinhexa.com;
set $upstream_keepalive false;
}
And your upstream (server 172.18.0.7:21347) being an HTTP endpoint.
Change your proxy_pass directive to use HTTP instead of HTTPS.
Not related, but using the same names (www.api.coinhexa.com) on both upstreams and server_names makes the configuration a bit confusing and possibly cause issues.
- Docker Compose wait for container X before starting Y
- How to use rules_oci to run an image local via Docker?
- Getting error "Get http://localhost:9443/metrics: dial tcp 127.0.0.1:9443: connect: connection refused"
- How to install pyarrow on an Alpine Docker image?
- Unable to push image to ACR from terraform - could not get auth config (the credentialhelper did not work or was not found):
- Docker returning Error 133: Failed to create temporary file with plenty of disk space
- Database's backup location that created by Azure Data Studio on Mac M1
- ubuntu wslregisterdistribution failed with error: 0x80041002
- System.InvalidOperationException: Failed to deploy distro docker-desktop to <localpath>: exit code: -1
- Installing specific version of NodeJS and NPM on Alpine docker image
- intellij idea wsl 2 can't load projet from \\WSL.LOCALHOST\[distrib]
- Docker "protocol not available" after context change
- How to configure different dockerfile for development and production
- Is it possible to convert the publish method of an App Service from Code to Docker?
- Jib create folder for temporary files and changing folder ownership
- Heroku: How to release an existing image in gitlab CI/CD?
- Access to full /proc in Docker container
- lookup registry-1.docker.io: no such host
- Nestjs doesn't handle request from nginx proxy_pass using docker-compose
- PackagesNotFoundError : The following packages are not available from current channels:
- Prisma openssl version issue
- why are alpine versions of node throwing an error that @jridgewell/gen-mapping module not found?
- Safe ways to specify postgres parameters for healthchecks in docker compose
- How to get a list of images on docker registry v2
- Docker is in volume in use, but there aren't any Docker containers
- PhpStorm inserts wrong client_host IP when trying to run debug session
- How to Properly Inject Service Account Credentials in Google Cloud Build for Cloud Run Deployment?
- Docker - COPY file from different working directory to another is not working
- starting container process caused: exec: "fastapi": executable file not found in $PATH: unknown
- docker login fails while using docker-credential-secretservice