I am trying to rewrite proxy_read_timeout
for a specific route that sends requests to Replicate that has longer response times.
If the AI model is cold, it takes more than 1 minute to respond and Nginx throws 504 timeout error
.
Below is my current nginx config. It throws Cannot POST /
error for all requests to /api/ai/
. All requests to /api/
are working fine.
What am I doing wrong?
location /api/ {
proxy_pass http://localhost:3000/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect /api/ /;
}
# Replicate
location /api/ai {
proxy_pass http://localhost:3000/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 120s;
proxy_read_timeout 120s;
proxy_send_timeout 120s;
}
In the first location block, requests to a URL like /api/foo/
are sent upstream to http://localhost:3000/foo/
effectively removing the /api
prefix.
Assuming that you wish the same transformation in the second location block, you need to make similar changes to both the location
and proxy_pass
statements.
For example:
location /api/ai {
proxy_pass http://localhost:3000/ai;
...
}
So requests to the URL /api/ai/
are sent upstream to http://localhost:3000/ai/