IP Address can't be Accessed Externally

EDIT:

I have done more research into my problem, and I'm starting to think that this is an issue with my isp. I have tried to run an Nmap scan on my server using a vpn, and Nmap thinks the host is either down or blocking the ping probes. When I run an Nmap scan from my network, I get the expected output of what ports are running. Could the isp be blocking access to the ip? If the isp is blocking access, should I call the isp to get the issue resolved?

Below is the original issue, which was about Nginx, which I don't think is the issue anymore. I'm still going to include this info in case it is helpful to solving the actual issue in any capacity

I currently have a server that I run Nginx on to host sites. This server has one ethernet connection directly to the modem, and three others that connect to a router on a different network. The server and Nginx have been working properly for months, but recently something has gone wrong with the server and Nginx. I'm not sure what the issue is, but I do know that Nginx is no longer serving the sites to anyone with an IP different from my modem. I can access the site from my server IP, or through a computer on the network with the routers the server is connected to. However, I cannot access the site on my phone using cellular or from someone else's network.

Whenever I try to access the site from a different IP, the error message is could not open the page because the server stopped responding.

I don't know if the issue is specifically related to Nginx, or if something is wrong with my modem or ISP. I've tried messing around with the Nginx config files, but nothing seems to work. Nginx is running the site, as I can access the site from my personal network. I do have a firewall on my server, but I allow all traffic through the right ports and the problem still exists when I disable the firewall.

Most other issues with Nginx serving, or lack thereof, talk about changing the server_name property in the config files. I have changed this value to _, the IP address of the server, and the domain name attached to the server, but none of these has worked.

Here is my Nginx file for one of my sites. I run php in the site, and Certbot handles the https.

server {
    root /home/user/www/site;
    
    index index.php index.html index.htm index.nginx-debian.html;

    server_name mydomain.com;

    location / {
        try_files $uri $uri/ =404;
    }
    
    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
    
        fastcgi_pass unix:/run/php/php7.4-fpm.sock;
    }

    listen 443 ssl; 
    ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem; 
    ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem; 
    include /etc/letsencrypt/options-ssl-nginx.conf; 
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; 
}

server {
    if ($host = mydomain.com) {
        return 301 https://$host$request_uri;
    } 

    server_name mydomain.com;
    listen 80;
    return 404; 
}

Here is my Nginx config file (I don't remember ever changing this file from the default, but something may have happened):

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 768;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    
    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; 

    ssl_prefer_server_ciphers on;

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    gzip on;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

    error_page 404 /404.html;
}

The issue I was having was that Nginx was only publishing my site to the local IP from my router, not the public IP from the modem. I went in to each of my available sites for Nginx (\etc\nginx\sites-available\*) and changed each listen XX; statement, where XX is the port number, to listen 12.34.56.78:XX;, where 12.34.56.78 was my public IP from the modem. This allowed Nginx to serve from the public IP.