Grok pattern to match URIPATH with optional URIPARAM
I want to use Grok Pattern for filtering out this
172.20.20.88 - - [10/Nov/2018:23:49:31 +0700] "GET /id/profile.pl?user=285&device=Bg3tlX HTTP/1.1" 502 852 "-" "Go-http-client/2.0" "0.009"
I am using COMMONAPACHELOG
%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)
I have tried %{URIPATH:request} and %{URIPARAM:request}. The result of request is still /id/profile.pl?user=285&device=Bg3tlX. My expectation is /id/profile.pl.
My reference is https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns
Your %{NOTSPACE:request} matches any 1 or more non-whitespace chars before HTTP/1.1" 502 85... as NOTSPACE pattern is \S+. So, it matches the whole /id/profile.pl?user=285&device=Bg3tlX substring.
You cannot use just URIPATH or URIPARAM, because you still need to match the rest of the input. You have to use both, but make URIPARAM optional after URIPATH by enclosing it within an optional non-capturing group, (?:...)?.
So, replace %{NOTSPACE:request} with
%{URIPATH:request}(?:%{URIPARAM:requestparam})?
^^^ ^^
Demo at https://grokdebug.herokuapp.com/:
- How to get matched pattern in ugrep
- Non greedy (reluctant) regex matching in sed?
- Split string into array of character strings
- Negative lookbehind in Python regular expressions
- How can I make my match non greedy in vim?
- REGEXP_SUBSTR in Snowflake, regex that will extract the string after the period (which occurs between 0-2 times)
- How to match letters only using java regex, matches method?
- Use dynamic (variable) string as regex pattern in dart
- PHP multibyte regex not working with UTF-8
- jQuery Validation Email Validation Issues
- How to remove numbers from string using Regex.Replace?
- URL - Get last part in PHP
- How to match Chinese date with or without month and time?
- How to make a RegExp that matches any beginning part of the RegExp?
- allMatches not matching several groups exactly
- Regex word boundary expressions
- How to use a pattern-list within a longer path name in a cronjob?
- php regex match similar to letters. Aka u=ü or ê=é=è=e
- Check for special characters (/*-+_@&$#%) in a string?
- Checking values row per row to assign proper work center to product
- MVC - Strip unwanted text from RSS feed
- Create a text field that block all special characters but enables a few
- How often does JavaScript recompile regex literals in functions?
- Copy files based on wild card file name; from SFTP to Blob storage - Azure Data Factory - Copy activity
- Recreating Regex Functions by Hand
- JavaScript to allow only numbers, comma, dot, backspace
- Regex.Match Uppercase
- How do I get the YouTube video ID from a URL?
- Regex to remove multiple characters
- Regex to match words of a certain length