I need to classify warnings for a python Google App Engine based application. I downloaded logs from GAE stackdriver. I think GAE Stackdriver Error Reporting at https://console.cloud.google.com/errors?time=P7D classifies (group and reduplicate) errors pretty well, but can it cluster warnings rather than errors? Is there any appropriate tool for that? I would preferably free offline software; I am rather looking for easiest/quick solution, but all the comments are appreciated.
PS. Well, "intelligent deduplication" might be not that important for warnings, yet even grouping by endpoint and warning text is somewhat of challenge as I new to this log format. So far I found a complicated recipe to manually massage a gae log file into an Excel table, yet I have no Excel and Libre Office is slow on large tables. Regretfully, Gnumeric dropped Windows support a while ago.
PPS. I am able to adjust log console page severity level to severity=WARNINGS, still I would like make sure I analyze all the cases without checking each and every log entry
What you want is possible but a little clumsier than you might want.
Error Reporting has an API. So you can just call it directly to report an error.
https://cloud.google.com/error-reporting/reference/rest/v1beta1/projects.events/report
Read this doc to see how the errors are grouped:
https://cloud.google.com/error-reporting/docs/grouping
So what you could do is write a Python logging handler that writes to the Error Reporting API on messages of warn, making sure that the same warn messages get grouped into the same group.
You can talk to the Error Reporting API via REST, gRPC, or via Stackdriver Logging entries formatted in a particular JSON way. We have Python client libraries to to Error Reporting unfortunately they might be annoying to get working on App Engine standard (not currently officially supported). There are also example Python logging handlers that talk to Stackdriver. So you could repurpose those handlers to do something along the lines of what you want. It'll be a bit of code to get that working though.