laravelamazon-web-servicesamazon-ec2redisamazon-elastic-beanstalk

Elastic Beanstalk CPU spike problem at specific times


I am experiencing CPU spike issues on my EBS instances. At the beginning of some minutes, the CPU usage of php-fpm occasionally spikes for a short time before returning to normal. I don't have any cron jobs in my Laravel project that could be causing this issue. I can't understand this behavior at all. When the CPU spikes, my response times decrease significantly. This problem occurs at exactly the same seconds of the minute (02,04,06).

When I log requests that take longer than 1 second in my Nginx access.log, these results appears:

14/Sep/2024:00:14:02 +0000  
14/Sep/2024:01:39:04 +0000  
14/Sep/2024:01:00:06 +0000] 
14/Sep/2024:01:13:02 +0000
14/Sep/2024:10:06:04 +0000
14/Sep/2024:09:50:02 +0000

For each time period, an average of 20-25 requests take longer than 1 second,My average response time is around 0.018 seconds, with 90% of requests completing in under 0.05 seconds so I am sure my php script is lightweight.

Project and system specifications:

Project and system specifications:

  1. Instance Type : T3 small (Amazon Linux 2023 AMI)
  2. Framework: Laravel
  3. Database: ElastiCache Redis (Redis database is not shared with any other project)

There is a video showing this behaviour, Please examine the change in idle CPU in the top command at 06:45:04 and 06:46:04: https://drive.google.com/file/d/1-iaTlionP52x5GyPm46maRjyOqKuVGep/view?usp=sharing

I cannot found problem yet.


Solution

  • Through all the run you have 1-5 processes running, the interesting part is what happens just before the spike.

    Your processes jump to 6-8 then going down to 1 then returning to complete jumping to 7 and returning to the normal numbers.

    Check your php-fpm.conf or www.conf file to see the values for:

    pm.max_children
    pm.start_servers
    pm.min_spare_servers
    pm.max_spare_servers
    

    Your conf might not fit your usage, probably too low, causing fpm spend a lot on balancing.

    I really question this option but you can also check your cache performance at those points.

    And lastly, t3 is not a good fit for real apps, check M6G or M7G, they are cheaper and give much better performance.