[SOLVED] aws nginx ec2 instance save access log to s3bucket

aws nginx ec2 instance save access log to s3bucket

I have an infra in aws that is composed of the following services:

load balancer
launch configuration
autoscaling
ec2 (nginx reverse proxy)

Everything works just smoothly without any particular issue but there is a little problem that is bothering me and I cannot find a easy way to solve.

Nginx is storing the logs under: /var/log/nginx/access.log

This is perfect, the issue is when the ec2 get terminated or scaled involving a new instance to take place, I lose all the logs from the nginx, which is not ideal.

So I wanted to store periodically the logs in an s3bucket to keep track of them and analyse them later on.

I saw some docs talking about log rotate but that didn't work for me. Is there any documentation or somebody can help me to understand if there is an easy solution to save the nginx logs somewhere else other than the volume storage?

Thank you so much for any guideline or help you can offer me

Solution

You could consider using Amazon CloudWatch Logs which uses an agent on the Amazon EC2 instance to send the logs to CloudWatch.

Alternatively, you could implement an Amazon EC2 Auto Scaling lifecycle hook that can send a signal when an Amazon EC2 instance is being terminated by Auto Scaling. You can write code that responds to this signal, copies the logs to S3 and then tells the hook to proceed with terminating the EC2 instance.