logstashlogstash-configurationoktalogstash-filterelastic-common-schema

How to translate Okta System Log records into Elastic Common Schema version 1.5 using logstash pipeline configuration

I have an Okta instance which I get system logs out of using logstash-input-okta_system_log plugin for Elastic Logstash.

The plugin works just fine. What I want is to translate the logs into Elastic Common Schema using a Logstash pipeline configuration. I can do that, but to be frank it is such a daunting task mapping,mutating,renaming the fields.

Now I am wondering if anyone has done this before and willing to share their filters?

I am not 100% sure if this goes against StackOverflow spirit, which I am sure many people will take issue with.

I have started working on it, if this is not something someone has done before I will post my solution as an answer for people looking for the same thing in the future.

I haven't found anything searching the Internet. Looking forward to hear from someone who has already done this.

filter {
  mutate {
    rename =>  {"displayMessage" => "message"}
    .
    .
    .

  }
}
Solution

  • For anyone interested, Elastic is releasing new Filebeat modules in a few weeks including one for Okta, which reads Okta system logs via the API and does the mapping to ECS.

    That's going to be what I will be using.

    Find details in the documents that are yet to be released: https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-module-okta.html