elasticsearchgrokingest

Grok processor in ingest pattern adds escaped double quotes on save

Im using elastic "Stack management" -> "Ingest pipelines" -> "Edit pipeline". In "Grok" processor fill fields "Pattern" with ^%{MESSAGE} and "Pattern definitions (optional)" with

{
    "GREEDYDATA": "(.|\n|\t)*",
    "MESSAGE": "%{GREEDYDATA:message}"
}

After pressing "Update" i saw that saved json in "Pattern definitions (optional)" becomes like this:

{
    "GREEDYDATA": """(.|
|   )*""",
    "MESSAGE": "%{GREEDYDATA:message}"
}

And my pattern not working. Its must be valid for any text wihthout qoutes, but no. In grok debug pattern works. Im tries escaping new lines and tabs, but it doesn't help. Why there quotes appears on save?

Solution

  • You need to save your pattern by escaping the newline and tabulation characters, like this:

    {
  "GREEDYDATA": "(.|\\n|\\t)*",
  "MESSAGE": "%{GREEDYDATA:message}"
}

    After you save it, it will look like this

    {
  "GREEDYDATA": """(.|\n|\t)*""",
  "MESSAGE": "%{GREEDYDATA:message}"
}

    So instead of containing verbatim newline and tabulation characters, your pattern will contain \n and \t instead.