Cannot integrate Nginx with SSLH

I am currently in the process of setting up a web server at my home.

I have port 443 and 80 open.

I am trying to integrate nginx but I am having some problems and I am running into this error: SSL handshake failed Error 525

Here is my current setup: I have SSLH running, so I can either connect with ssh through port 443, or I can simply visit my website thats also running on port 443. In other words, I am multiplexing port 443 for either ssh of my website. Here is my sslh config:

sourced by /etc/init.d/sslh
Run=yes
# binary to use: forked (sslh) or single-thread (sslh-select) version
# systemd users: don't forget to modify /lib/systemd/system/sslh.service
DAEMON=/usr/sbin/sslh
DAEMON_OPTS="--user sslh --listen 0.0.0.0:443 --ssh 127.0.0.1:22 --ssl 127.0.0.1:8443 --pidfile /var/run/sslh/sslh.pid"

I then have nginx running on 8443, here is the config:

server {
    listen 8443 ssl http2;
    listen [::]:8443 ssl http2;
    server_name domain.xyz www.domain.xyz;
    ssl_certificate         cert.pem;
    ssl_certificate_key     cert.key;
    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Finally, I have my web node js app running on port 3000

const https = require('https');
const fs = require('fs');

const options = {
  key: fs.readFileSync('cert.key'),
  cert: fs.readFileSync('cert.pem')
};
https.createServer(options, (req, res) => {
  res.writeHead(200);
  res.end('Website !');
}).listen(3000, '127.0.0.1', () => {
  console.log('Server running on https://localhost');
});

I don’t understand why this setup doesn’t work. If I get rid of nginx and I simply forward to 127.0.0.1:3000 from the sslh config, it works perfectly.

Solution

The SSL handshake error (Error 525) typically indicates an issue with SSL/TLS certificates or connection settings. There are a few potential issues in your current setup:

SSL Conflict Between Nginx and Node.js:

Nginx accepts SSL connections on port 8443 and then forwards these connections to your Node.js application. However, your Node.js application also uses SSL on its own. This could cause an SSL conflict between Nginx and Node.js.
As a solution, you can run your Node.js application over HTTP and let Nginx handle the SSL termination. In this case, run your Node.js application on port 3000 over HTTP and use Nginx as the SSL termination point.

To run your Node.js application over HTTP:

const http = require('http');

http.createServer((req, res) => {
  res.writeHead(200);
  res.end('Website !');
}).listen(3000, '127.0.0.1', () => {
  console.log('Server running on http://localhost:3000');
});

Nginx SSL Configuration:

Ensure that the ssl_certificate and ssl_certificate_key directives in your Nginx configuration point to the correct certificate files. Check the paths and permissions of the certificate files.
Also, make sure that Nginx is configured with the correct SSL protocols and ciphers. For example:


ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;

SSLH Configuration:

Your SSLH configuration looks correct, but ensure that Nginx is listening on port 8443 and that SSLH is correctly forwarding to this port.

After making these changes, check if Nginx and your Node.js application are working correctly. If the issue persists, you can examine the Nginx and Node.js logs for more information.