Back in 2009 we relied on IRC to bootstrap the network, so every node would connect to Freenode (later LFnet) and would join a channel. Their nicknames were their encoded public IP address.

Nowadays the Bitcoin Core client, and many other implementations, rely on DNS seeds. DNS seeds are special DNS servers that are configured to return a number of randomly selected nodes from the network. The operators of the DNS seeds also run crawlers to enumerate the publicly reachable nodes that are to be returned by the seeds.

The seeds that are currently included in the Bitcoin Core client are:

bitcoin.sipa.be
dnsseed.bluematt.me
dnsseed.bitcoin.dashjr.org
seed.bitcoinstats.com
bitseed.xf2.org
bitcoin.jonasschnelli.ch

If you send a request to any of these servers they will return a number of random IPs that are known to run Bitcoin on port 8333:

dig seed.bitcoinstats.com +short
71.19.155.244
173.254.232.51
45.79.97.30
198.252.112.64
35.128.8.141
108.17.18.165
98.208.76.134
8.29.28.12
52.62.2.124
96.234.214.85
47.89.24.56
212.164.215.159
52.62.42.229
68.52.96.191
115.66.205.171
24.250.16.39
201.43.160.155
5.3.253.18
100.40.179.172
50.135.169.181
186.149.249.18
101.201.44.207
96.35.97.46
124.188.118.196
82.8.4.79

Besides the DNS seeds, the Core client also has a static list of IPs to try first and it will cache any previously contacted peers in a local database in order to reconnect without having to query the DNS seeds.

(Disclaimer: I am the operator of one of the DNS seeds)