azurelogstashgrayloggraylog2gelf

Configuring Azure Activity Logs to Graylog / ELK


I have exported Azure Activity Logs to Blob Storage. I am using Logstash 8.7.1 version along with gelf output using docker-compose to ship these logs to Graylog using GelfUDP. For Input of Logstash, using azure_blob_storage plugin of Logstash.

However Logstash is unable to send these logs to Graylog.

input {
     azure_blob_storage {
          connection_string => "DefaultEndpointsProtocol=https;AccountName=<BLOB_NAME>;AccountKey=<BLOB_ACCOUNT_KEY>;EndpointSuffix=core.usgovcloudapi.net"
         container => "insights-activity-logs"
         registry_create_policy => "start_over"
         codec => "json"
         addall => true
         path_filters => ['**/*.json']
         addfilename => true
         prefix => "resourceId=/"
         # Possible options: `do_not_break`, `with_head_tail`, `without_head_tail`
         interval => 5
     }
 }

filter {
    json {
        source => "message"
    }
    mutate {
        add_field => {"short_message" => ["This is short message"]}
        add_field => { "host" => "127.0.0.1" }
    }
    date {
        match => ["unixtimestamp", "UNIX"]
    }
}

output {
    gelf {
        host => "127.0.0.1"
        port => 12201
        protocol => "UDP"
        codec => "json"
    }
}

I also checked with stdout output. it is showing all messages from blob. What can be the issue? Or how can I ship Azure Activity logs to Graylog?


Solution

  • Fixed this issue by configuring Gelf TCP Output and in Graylog Gelf TCP Input.

    Working output code:

    output {
        gelf {
            host => "127.0.0.1"
            port => 12201
            protocol => "TCP"
            codec => "json"
        }
    }