logstashgsublogstash-grok

How to remove \r in Logstash


I need to parse /etc/passwd files for security reasons looking for expired or fake accounts. After parsing one example file with the following filter:

filter {
    grok {
        match => {"message" => [ "%{WORD:username}:%{WORD:password}:%{NUMBER:userid}:%{NUMBER:groupid}:%{DATA:useridinfo}:%{DATA:homedir}:%{GREEDYDATA:command-shell}" ]
        }
    }
    mutate {
            remove_field => [ "@version", "log", "host", "@timestamp", "event", "message" ]
    }
    mutate {
        gsub => [ "command-shell", "\\r", "" ]
        }
}

The output is right, except for the "command-shell" field, which has a '\r' in each and every line, despite the gsub inside the mutate filter plugin.

Example:

{"useridinfo":"guest","userid":"405","groupid":"100","homedir":"/dev/null","username":"guest","password":"x","command-shell":"/sbin/nologin\r"}

{"useridinfo":"nobody","userid":"65534","groupid":"65534","homedir":"/","username":"nobody","password":"x","command-shell":"/sbin/nologin\r"}

The original example lines were:

guest:x:405:100:guest:/dev/null:/sbin/nologin

nobody:x:65534:65534:nobody:/:/sbin/nologin

I want to remove that \r and nothing seems to work, though there is no error in Logstash and it ends ok.

Could you please help me removing that? I'm new to Logstash, I tried DATA instead of GREEDYDATA but it didn't work in Grok Debugger, so I left GREEDYDATA instead, maybe that is the problem?

Thank you.

EDIT:

**Thank you for answering my question.

The first one leaves the \r unchanged:

{"homedir":"/","userid":"65534","username":"nobody","password":"x","command-shell":"/sbin/nologin\r","groupid":"65534","useridinfo":"nobody"}

The second option gives me (always following the same example) this:

{"command-shell":"/sbin/nologin\r","useridinfo":"nobody","groupid":"65534","userid":"65534","password":"x","username":"nobody","homedir":"/"}

So sadly there is no change at all.

Please tell me if you need more info on this, I need to solve it and I'm not getting to a solution.**


Solution

  • If your field contains the string "\r" you would remove that using

    mutate { gsub => [ "message", "\\r", "" ] }
    

    If, as in your case, your field contains a carriage-return character (Ctrl/M) then you can remove it using

    mutate { gsub => [ "message", "\r", "" ] }