elasticsearchkibana-6

Elasticsearch time-range query and data


I am struggling to formulate the right API search call for Elastic Search that will ask for ipv4address that I want in last 1 hour.

First attempt:

curl -X GET "localhost:9200/ipaddresses/_search" -H 'Content-Type: application/json' -d'
     {
       "query": {
         "match": {
           "ipv4address": {
             "query": "50.167.71.25"
           }
         }
       },
       "range": {
         "@timestamp": {
           "gte": "now-1h",
           "lt": "now"
         }
       }
     }
     '

{"error":{"root_cause":[{"type":"parsing_exception","reason":"Unknown key for a START_OBJECT in [range].","line":10,"col":12}],"type":"parsing_exception","reason":"Unknown key for a START_OBJECT in [range].","line":10,"col":12},"status":400}

Second attempt:

curl -X GET "localhost:9200/ipaddresses/_search" -H 'Content-Type: application/json' -d'
{
   "query": {
     "match": {
       "ipv4address": {
         "query": "50.167.71.25"
       }
     }
   },
   "fields": {
    "range": {
     "@timestamp": {
      "gte": "now-1h",
      "lt": "now"
     }
   }
  }
 }
 '

{"error":{"root_cause":[{"type":"parsing_exception","reason":"Unknown key for a START_OBJECT in [fields].","line":10,"col":14}],"type":"parsing_exception","reason":"Unknown key for a START_OBJECT in [fields].","line":10,"col":14},"status":400}

What I have in Kibana:

{
  "_index": "ipaddresses",
  "_type": "default",
  "_id": "TJdvR2UB9sEBYW4CrElF",
  "_version": 1,
  "_score": null,
  "_source": {
    "tags": [
      "blocked",
      "ipv4_address",
    ],
    "@version": "1",
    "@timestamp": "2018-08-17T10:30:25.118Z",
    "ipv4_metadata": {
      "host": "elk",
      "name": "blocks",
      "response_message": "OK",
      "code": 200,
      "times_retried": 0,
      "runtime_seconds": 0.066403,
      "response_headers": {
        "connection": "keep-alive",
        "x-frame-options": "sameorigin",
        "last-modified": "Fri, 17 Aug 2018 10:28:06 GMT",
        "keep-alive": "timeout=20",
        "date": "Fri, 17 Aug 2018 10:28:20 GMT",
        "content-type": "text/plain; charset=UTF-8",
        "server": "nginx/1.12.2",
        "transfer-encoding": "chunked",
        "etag": "W/\"5c7c5-5739f03f2997f\"",
        "cache-control": "public"
      }
    },
    "ipv4address": "50.167.71.25",
    "message": "50.167.71.25"
  },
  "fields": {
    "@timestamp": [
      "2018-08-17T10:30:25.118Z"
    ]
  },
  "sort": [
    1534501825118
  ]
}

What is wrong with the query? What if I would like to look as well for a "tag" field that equals blocked?

Please help me to connect the dots.


Solution

  • This query will return the documents from last 1 hr:

    {
       "query": {
         "range": {
           "@timestamp": {
             "gte": "now-1h",
             "lt": "now"
           }
         }
       }
     }
    

    This query will return the documents where tag is blocked and is from last 1hr:

    {
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "tags": "blocked"
              }
            },
            {
              "range": {
                "@timestamp": {
                  "gte": "now-1h",
                  "lte": "now"
                }
              }
            }
          ]
        }
      }
    }
    

    You can limit the data to be returned using _source.

    This query will only return the ipv4address:

    {
      "_source": "ipv4address", 
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "tags": "blocked"
              }
            },
            {
              "range": {
                "@timestamp": {
                  "gte": "now-1h",
                  "lte": "now"
                }
              }
            }
          ]
        }
      }
    }
    

    If you want to apply more queries have a look at this.