goaccess

GoAccess custom forwarded log parsing


I am currently using goaccess-1.0.2. I have installed it on an Amazon Linux box. The box which it resides has customized logs that were forwarded from an Apache WebApp Server. What I have tried to accomplish but can't seem to figure out is how to get GoAccess to parse our customized log.

Here is an example of the custom forwarded WebApp Log entry:

Jun 24 00:00:41 directory1 httpd-access: 55.117.170.95 www.URLaddress.com - [24/Jun/2016:00:00:41 -0700] "GET /sites/all/themes/somthing_on_demand/js/fancybox/jquery.fancybox-1.3.4.css HTTP/1.1" 304 - "ht
tps://www.IPaddress.com/my_account/yum" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36" "SESSb9948a0b21e4d377a7d82f6adbf86c91=l
on7pgjlikml7q4tq954ejiao1; cookie_js=1; __utma=23285183.1119616966.1452095139.1468883973.1468963151.39; __utmb=23285183.500.10.1468963151; __utmc=23285183; __utmz=23285183.1468963151.39.39.utmcsr=fyi.URLaddress.com|utm
ccn=(r/INFOSEC-MAXLEN-256" "-" 57630

Here are a few log-formats I have tried:

log-format %^ %^ %^ "%h %^ %u %t \"%r\" %>s %b \"%R\" \"%u\" \"%^\" \"%^\" %D"

log-format "%h %{Host}i %{SSL_CLIENT_S_DN_CN}x %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{SHORT_COOKIE}e\" \"%{X-Forwarded-For}i\" %D"

log-format "%h %{Host}i %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{SHORT_COOKIE}e\" \"%{X-Forwarded-For}i\" %D"

I thought I would ignore the date and time format using %^ then use date format %m %d and time format %T .

I am very new at this and could really use help. Thank you for your feedback in advance.


Solution

  • Please try this, it works for me:

    goaccess -f access.log --log-format='%^:%^:%^: %h %v %^[%d:%t %^] "%r" %s %b "%R" "%u" "%^" "%^" %D' --date-format='%d/%b/%Y' --time-format='%T'