I want to build a service similar to ngrok and I'm trying to understand the architecture behind ngrok's subdomain management for their tunneling service. From what I can see, ngrok provides unique subdomains (like abcde.ngrok.io) for each tunnel, but I'm puzzled about how they manage this at scale.
Specifically:
And the main question: is there a way to implement a similar system for a smaller scale project? What would be the key components?
Any insights into the potential architecture or best practices for implementing such a system would be greatly appreciated! Thanks!
Does each tunnel have a separate IP address? If so, how come ngrok has such enormous number of IPs?
No, the tunnel addresses are shared between users.
How do they dynamically add new DNS records for their IPs?
First, wildcard subdomains. DNS supports records named like *.example.com which would automatically cover any (one-level) subdomain of .example.com that isn't specifically defined otherwise.
Second: DNS records, by analogy with HTTP URLs, don't need to be added somewhere, as there is no central database of subdomains; instead, that information is provided solely by Ngrok's own nameservers. So just like an HTTP webapp can dynamically respond for various URLs through code, it is possible to write a DNS server that dynamically responds for various subdomains.
If different subdomains are managed by the same IP address, how does ngrok understand that I connected to the a.ngrok.io, not b.ngrok.io if they both have the same IP address? I know, there are technologies like SNI, but how does it work then if I make TCP tunnel without TLS encryption?
For plain TCP tunnels, they don't really know that. As far as I understand their system, they only use TCP port numbers to distinguish between tunnels, as you never actually get a whole subdomain – you only get a single TCP port on that IP address.
And the main question: is there a way to implement a similar system for a smaller scale project? What would be the key components?
Depends on scale. At really small scale (like maybe 0–3 users), even standard SSH servers can provide the same kind of tunneling using ssh -R (and Ngrok even offers the same kind of SSH-style interface to their custom backend).
The basic code for such a service would begin kind of like a regular TCP proxy (rinetd, haproxy, nginx) – you open a listening socket, and for every received connection you make an outgoing connection and make a poll() loop that copies data from one socket to the other.
To make it work like Ngrok's service, though (where the entire idea is that Ngrok can't connect to the backend; the backend has to connect to Ngrok), you'd need to change it so that the proxy is listening both for 'client' connections and 'agent' connections on two different listeners, and pairing them up in a similar way. (Similar to how socat tcp-l:1234 tcp-l:2345 would work.)
This would be limited to one client at a time, so the next step from that would be to change the 'agent-proxy' connection so that it can multiplex data from multiple clients (similar to e.g. how SSH or QUIC have several distinct "streams" of data inside the TCP connection) – and make the 'agent' demultiplex them.
For example, the proxy now accepts one 'agent' connection, and for each received 'client' connection it sends a command to the agent like "client #5 connected – open a new connection to the backend, please" and "send this chunk of data over backend connection #5", and so on. (This is very much like how ssh -R works, too.)
The rest is just making the proxy more flexible, e.g. making it automatically allocate a new 'client' listener whenever it receives an 'agent' connection, implementing keep-alive checks for NAT, etc.