[SOLVED] Why can I get an SSL certificate for a website, but it remains inaccessible?

Why can I get an SSL certificate for a website, but it remains inaccessible?

I am having problems serving my website online.

I have a server running Virtualmin
My register has told me that the A and CNAME records have been propagated correctly.
I have made sure that the external IP is correct in my A record and in Virtualmin.
I can access the website locally via IP and via domain name.
I have forwarded ports 80 and 443 through both routers to the correct machine.
I have gotten an SSL certificate from Let's Encrypt for the A record and the www CNAME.

Trying to access the website via domain name gets me nowhere.
Browsing my external.IP:80 or external.IP:443 I get a warning message telling me that the "connection to this site is not secure: - Domain in certificate does not mathc the site domain. Visit site anyway?" I select yes and the site doesn't load anyways, but the URL changes to my local IP.
Accessing the site through via domain name in my local network will load it, but immediately change the url to the local IP.

Any ideas on what might be going wrong? I have run out of ideas. Thanks in advance.

Solution

So, as @ceejayoz suggested, it was down to Wordpress.

I just had to change the URL under Settings > General.