I have a config file for logrotate for my ModSecurity3 logs (on Ubuntu 22.04 + Nginx Web Server)
/etc/logrotate.d/modsec
-rw-r--r-- 1 root root 162 Nov 2 23:21 modsec
/var/log/modsec/modsec_audit.log
{
size 1M
rotate 7
missingok
compress
delaycompress
notifempty
}
This is set up as a test, to rotate the logs whenever they grow larger than 1M.
In the modsec logs directory, I have this:
ubuntu@nginx:/var/log/modsec$ ls -al
drwxr-xr-x 2 root root 4096 Nov 2 23:20 .
drwxrwxr-x 13 root syslog 4096 Nov 2 23:19 ..
-rw-r--r-- 1 root root 47744907 Nov 2 23:27 modsec_audit.log
The size is growing larger than 1M and they are not rotating. I tried to debugging this and got this:
ubuntu@nginx:/var/log/modsec$ sudo logrotate -d /etc/logrotate.d/modsec
WARNING: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.
reading config file /etc/logrotate.d/modsec
Reading state from file: /var/lib/logrotate/status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Handling 1 logs
rotating pattern: /var/log/modsec/modsec_audit.log
1048576 bytes (7 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/modsec/modsec_audit.log
Now: 2022-11-02 23:32
Last rotated at 2022-11-02 23:20
log needs rotating
rotating log /var/log/modsec/modsec_audit.log, log->rotateCount is 7
dateext suffix '-20221102'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /var/log/modsec/modsec_audit.log.1 does not exist
renaming /var/log/modsec/modsec_audit.log.7.gz to /var/log/modsec/modsec_audit.log.8.gz (rotatecount 7, logstart 1, i 7),
renaming /var/log/modsec/modsec_audit.log.6.gz to /var/log/modsec/modsec_audit.log.7.gz (rotatecount 7, logstart 1, i 6),
renaming /var/log/modsec/modsec_audit.log.5.gz to /var/log/modsec/modsec_audit.log.6.gz (rotatecount 7, logstart 1, i 5),
renaming /var/log/modsec/modsec_audit.log.4.gz to /var/log/modsec/modsec_audit.log.5.gz (rotatecount 7, logstart 1, i 4),
renaming /var/log/modsec/modsec_audit.log.3.gz to /var/log/modsec/modsec_audit.log.4.gz (rotatecount 7, logstart 1, i 3),
renaming /var/log/modsec/modsec_audit.log.2.gz to /var/log/modsec/modsec_audit.log.3.gz (rotatecount 7, logstart 1, i 2),
renaming /var/log/modsec/modsec_audit.log.1.gz to /var/log/modsec/modsec_audit.log.2.gz (rotatecount 7, logstart 1, i 1),
renaming /var/log/modsec/modsec_audit.log.0.gz to /var/log/modsec/modsec_audit.log.1.gz (rotatecount 7, logstart 1, i 0),
log /var/log/modsec/modsec_audit.log.8.gz doesn't exist -- won't try to dispose of it
renaming /var/log/modsec/modsec_audit.log to /var/log/modsec/modsec_audit.log.1
It seems that logrotate knows the log needs to be rotated, but it's not doing it automatically... Or I'm doing something wrong. How do I get this working?
Intent: My ModSecurity logs are growing too large, so in the end, I'd like to limit the number of logs to 7, no larger than 1GB each. If it grows larger than that, rotate the log and delete the oldest log, otherwise just rotate daily.
Was able to find a solution after looking at a variety of sources. Part of the problem was related to Spiderlab's ModSecuritiy-Nginx connector not writing to a new log after rotation. This config file (/etc/logrotate.d/modsec) eventually worked:
/var/log/modsec/*.log {
size 1G
missingok
rotate 7
compress
delaycompress
notifempty
copytruncate
sharedscripts
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi \
endscript
postrotate
invoke-rc.d nginx rotate >/dev/null 2>&1
endscript
}
This is basically a copy of the default Nginx logrotate config for error.log/access.log logs with a couple of changes. The key was to replace the "create" line with "copytruncate". This takes the modsec_audit.log, copies it to modsec_audit.log.1 and then truncates the modsec_audit.log into a blank file - so the new data continues to write to it.
The second part of the solution to limit the file sizes is to create a cronjob:
@hourly /usr/sbin/logrotate /etc/logrotate.d/modsec
This runs the logrotate config file hourly and checks the log size. If the log size is less than 1G, nothing will happen, otherwise if the size is larger than 1G, the logs will rotate. Without the cronjob, the default logrotate config only gets run daily (only checks the file size daily). This is why my log grew to over 20GB without size checks...