I've got the following rsyslog conf and the below log message I'm receiving. I'm extracting the timestamp from the log message using regex but since it's a not so nice format, I want to convert the timestamp to rfc3339.
I find rsyslog's documentation is missing a lot of things for someone starting fresh. Is this something which can be done within the template? I appreciate any hints/clues how to achieve this.
/path/to/log/file.log
11955 - [Mon Apr 6 20:40:03 2023] [Info ] This message can contain anything [d54d13fa-4657-4891-f99d08674ee]
/etc/rsyslog.d/mylog.conf
module(load="imfile")
input(type="imfile" tag="mylog" file="/path/to/*/file.log")
template(name="jsonFormat" type="list") {
property(outname="timestamp" name="msg" regex.expression="^[^[]*\\[([^]]*)\\]" regex.type="ERE" regex.submatch="1" format="jsonf")
}
if ($syslogtag == "mylog") then {
action(type="omfile" file="/path/to/output/file.log" template="jsonFormat")
}
RainerScript has some functions you can use to manipulate the input line with. For example,
template(name="myformat" type="string" string="%$.date% %msg%\n")
set $.date = re_extract($rawmsg, "\\[... (.{15})", 0, 1, "");
if ($.date != "") then {
set $.unixtime = parse_time($.date);
if($.unixtime != 0) then {
set $.date = format_time($.unixtime, "date-rfc3339");
action(type="omfile" file="output" template="myformat")
}
}
The re_extract() function works like the regex.expression in your
template property. Here it finds the [, skips the 3 character weekday,
and captures the next 15 characters, omitting the year as that is not part
of RFC3164. The year is assumed to be approximately "this year".
The returned value is saved in a local variable of your choice, $.date. Note the obligatory ; at the end of lines beginning set.
If
the match worked, parse_time() is used to convert it from RFC3164 to Unix
seconds-from-the-epoch. If this worked, format_time() converts it to an
RFC3339 string. The variable is used in a template just like any property with
%$.date% in a string or property(name="$.date" ...).