i have deployment using container at ECS that contain static web and nginx for reverse proxy. I'm using nginx:alpine as base image
Use Nginx to serve a static website and as a reverse proxy for another service inside ECS, which is behind a VPC.
I have an AWS load balancer in front of ECS.
/etc/nginx/nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
resolver 8.8.8.8 valid=60s;
resolver_timeout 5s;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
upstream backend {
server api.com:443;
keepalive 32;
}
# proxy_read_timeout 10s;
# proxy_connect_timeout 10s;
# proxy_send_timeout 10s;
# send_timeout 10s;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
/etc/nginx/conf.d/default.conf
server {
listen 3000;
server_name web.com;
location / {
root /usr/share/nginx/html;
index index.html;
try_files $uri /index.html;
}
location /intools/ {
client_max_body_size 50M;
proxy_pass https://api.com;
proxy_set_header Host api.com;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Authorization $http_authorization;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_connect_timeout 10s;
proxy_send_timeout 10s;
proxy_read_timeout 10s;
send_timeout 10s;
access_log /dev/stdout;
error_log /dev/stderr debug;
}
location /api/ {
proxy_pass https://api.com;
proxy_set_header Host api.com;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Authorization $http_authorization;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_connect_timeout 10s;
proxy_send_timeout 10s;
proxy_read_timeout 10s;
send_timeout 10s;
access_log /dev/stdout;
error_log /dev/stderr debug;
}
}
My problem is that the first deployment works fine, but after a few days, Nginx starts behaving strangely. The routing, which is supposed to point to api.com, unexpectedly redirects to random unknown hostnames. This issue happens frequently. Every time I restart the ECS container, it works fine for a while, but then it starts pointing to another random hostname again.
Has anyone ever experience this?
multiple response that i got first
{
"Message": "User: anonymous is not authorized to perform: es:ESHttpGet because no resource-based policy allows the es:ESHttpGet action"
}
second
404 Not Found
nginx
Turns out I needed to add a DNS resolver to AWS DNS under the server block:
server {
resolver 169.254.169.253 valid=10s;
}
After that, the issue never happened again.