linuxserverhistogramcpudiagnostics

Trying to find out why a Linux server had high cpu usage in the past


enter image description here

One of my linux servers in the cloud had very high cpu usage yesterday, then the issue somehow disappeared by itself.

If there a way to find out which process was taking all the cpu power yesterday?

For example, I want to find out which process was using the most cpu yesterday during 10AM~11AM, is this achievable?


Solution

  • If you were running super-detailed logging, you might have the info recorded. But probably not; that kind of logging would take a lot of space (not just load average every few minutes, but a full snapshot of top output.)

    So probably not.

    If it was a process that's still running, it's total CPU time might be higher than normal, but you don't know that. (Look at the "time" field in top. But that's just total since the process started, with no record of when it happened.)

    If it was something that ran from cron (and then exited), you could look at cron logs.

    Or in general look for any log activity from around then; some system processes do end up logging things in the system logs. (journalctl with options to look around that time window.) But that will probably just give you hints about what might have started around then.


    Another source of possible hints could be mod times on files. If you find some files that were modified around that time, that might remind you of something you'd set up to run occasionally.

    find / -xdev -mmin -$((25*60)) -mmin +$((24*60))
    

    would I think print all files (on the root filesystem) that are older than 24 hours old (24*60 minutes), but younger than 25 hours old. (Relative to now; -daystart changes that I think.)