nginxnginx-reverse-proxynginx-confignginx-locationreplicate

Nginx location specific timeout


I am trying to rewrite proxy_read_timeout for a specific route that sends requests to Replicate that has longer response times. If the AI model is cold, it takes more than 1 minute to respond and Nginx throws 504 timeout error.

Below is my current nginx config. It throws Cannot POST / error for all requests to /api/ai/. All requests to /api/ are working fine.

What am I doing wrong?

        location /api/ {
            proxy_pass http://localhost:3000/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_redirect /api/ /;

        }
          # Replicate
        location /api/ai {
            proxy_pass http://localhost:3000/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            proxy_connect_timeout 120s;
            proxy_read_timeout 120s;
            proxy_send_timeout 120s;
        }

Solution

  • In the first location block, requests to a URL like /api/foo/ are sent upstream to http://localhost:3000/foo/ effectively removing the /api prefix.

    Assuming that you wish the same transformation in the second location block, you need to make similar changes to both the location and proxy_pass statements.

    For example:

    location /api/ai {
        proxy_pass http://localhost:3000/ai;
        ...
    }
    

    So requests to the URL /api/ai/ are sent upstream to http://localhost:3000/ai/