I have a nginx custom log file which is like :
{ "timestamp": "2015-12-20T12:30:40+03:30", "remote_addr": "80.253.148.194", "remote_user": "-", "body_bytes_sent": "0", "request_time": "0.000", "status": "304", "request": "GET /repository/9/13350-000036.png HTTP/1.1", "request_method": "GET", "http_referrer": "http://domain.ltd/profile/category/1425", "http_user_agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0" }
{ "timestamp": "2015-12-20T12:30:40+03:30", "remote_addr": "80.253.148.194", "remote_user": "-", "body_bytes_sent": "0", "request_time": "0.000", "status": "304", "request": "GET /repository/9/13349-000018.png HTTP/1.1", "request_method": "GET", "http_referrer": "http://domain.ltd/profile/category/1425", "http_user_agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0" }
{ "timestamp": "2015-12-20T12:27:24+03:30", "remote_addr": "79.175.174.23", "remote_user": "-", "body_bytes_sent": "139008", "request_time": "0.298", "status": "206", "request": "GET /repository/9/56766da1202f7.mp4 HTTP/1.1", "request_method": "GET", "http_referrer": "-", "http_user_agent": "Lavf/56.4.101" }...
it consists 3792061 rows.
i've setup the log format and it works fine , but after the job is done there's some data missing like there's no /9/ files in the report.
HERE is my goaccess report.
and my command is :
# goaccess -o json --static-file=.mp4 --static-file=.MP4 --ignore-panel=VISITORS --ignore-panel=REQUESTS --ignore-panel=NOT_FOUND --ignore-panel=HOSTS --ignore-panel=OS --ignore-panel=BROWSERS --ignore-panel=VISIT_TIMES --ignore-panel=VIRTUAL_HOSTS --ignore-panel=REFERRERS --ignore-panel=REFERRING_SITES --ignore-panel=KEYPHRASES --ignore-panel=GEO_LOCATION --ignore-panel=STATUS_CODES --sort-panel=REQUESTS_STATIC,BY_BW,DESC --debug-file debug.log --invalid-requests invalid.log -f ./access.json > data.json
invalid requests file has 40 rows which is ignorable.
But afterwards i've split my files and find the rows with /9/ in it # cat access.json | fgrep '/9/' ./access.json > newlog.json
i execute exactly the same command in goaccess for this log and i get This output which is fine.
Got stuck there, any help is appreciated
It only displays the top 366 items.
each active window has a total of 366 items. Eventually this will be customizable. These 366 items are all available by default in the CSV and JSON exports, and as an expandable panel in the HTML report (upper-right corner).