dockerdocker-composelogstashelastic-stackkibana-5

ELK Docker -- Logstash not working properly


I am trying to run the ELK stash using the popular Docker image on DockerHub, seep/elk.

In my project dir, I have the following two files:

docker-compose.up (just want to see if logstash works, so I'm reading from stdin and writing to stdout rather than elasticsearch):

input { stdin {} }
output { stdout {} }

logstash.conf:

elk:
  image: sebp/elk
  ports:
    - "5605:5601"
    - "9200:9200"
    - "9300:9300"
    - "5044:5044"
  volumes:
    - /path/to/project/dir/logstash.conf:/usr/share/logstash/config/logstash.conf

When I run docker-compose up elk, the following stack trace is displayed:

elk_1  |  * Starting periodic command scheduler cron
elk_1  |    ...done.
elk_1  |  * Starting Elasticsearch Server
elk_1  |    ...done.
elk_1  | waiting for Elasticsearch to be up (1/30)
elk_1  | waiting for Elasticsearch to be up (2/30)
elk_1  | waiting for Elasticsearch to be up (3/30)
elk_1  | waiting for Elasticsearch to be up (4/30)
elk_1  | waiting for Elasticsearch to be up (5/30)
elk_1  | waiting for Elasticsearch to be up (6/30)
elk_1  | waiting for Elasticsearch to be up (7/30)
elk_1  | waiting for Elasticsearch to be up (8/30)
elk_1  | waiting for Elasticsearch to be up (9/30)
elk_1  | waiting for Elasticsearch to be up (10/30)
elk_1  | waiting for Elasticsearch to be up (11/30)
elk_1  | Waiting for Elasticsearch cluster to respond (1/30)
elk_1  | logstash started.
elk_1  |  * Starting Kibana5
elk_1  |    ...done.
elk_1  | ==> /var/log/elasticsearch/elasticsearch.log <==
elk_1  | [2018-08-11T17:34:41,530][INFO ][o.e.g.GatewayService     ] [pIJHFdO] recovered [0] indices into cluster_state
elk_1  | [2018-08-11T17:34:41,926][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pIJHFdO] adding template [.watches] for index patterns [.watches*]
elk_1  | [2018-08-11T17:34:42,033][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pIJHFdO] adding template [.watch-history-7] for index patterns [.watcher-history-7*]
elk_1  | [2018-08-11T17:34:42,099][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pIJHFdO] adding template [.triggered_watches] for index patterns [.triggered_watches*]
elk_1  | [2018-08-11T17:34:42,205][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pIJHFdO] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-6-*]
elk_1  | [2018-08-11T17:34:42,288][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pIJHFdO] adding template [.monitoring-es] for index patterns [.monitoring-es-6-*]
elk_1  | [2018-08-11T17:34:42,338][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pIJHFdO] adding template [.monitoring-beats] for index patterns [.monitoring-beats-6-*]
elk_1  | [2018-08-11T17:34:42,374][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pIJHFdO] adding template [.monitoring-alerts] for index patterns [.monitoring-alerts-6]
elk_1  | [2018-08-11T17:34:42,431][INFO ][o.e.c.m.MetaDataIndexTemplateService] [pIJHFdO] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-6-*]
elk_1  | [2018-08-11T17:34:42,523][INFO ][o.e.l.LicenseService     ] [pIJHFdO] license [f28743a3-8cc3-46ad-8c75-7c096c7afaa7] mode [basic] - valid
elk_1  | 
elk_1  | ==> /var/log/logstash/logstash-plain.log <==
elk_1  | 
elk_1  | ==> /var/log/kibana/kibana5.log <==
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:kibana@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:elasticsearch@6.3.2","info"],"pid":247,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:xpack_main@6.3.2","info"],"pid":247,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:searchprofiler@6.3.2","info"],"pid":247,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:ml@6.3.2","info"],"pid":247,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:tilemap@6.3.2","info"],"pid":247,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:watcher@6.3.2","info"],"pid":247,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:license_management@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:index_management@6.3.2","info"],"pid":247,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:timelion@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:graph@6.3.2","info"],"pid":247,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:55Z","tags":["status","plugin:monitoring@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:searchprofiler@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:ml@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:tilemap@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:watcher@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:index_management@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:graph@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:security@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:grokdebugger@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:logstash@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["status","plugin:reporting@6.3.2","info"],"pid":247,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["info","monitoring-ui","kibana-monitoring"],"pid":247,"message":"Starting all Kibana monitoring collectors"}
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:34:57Z","tags":["license","info","xpack"],"pid":247,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active"}
elk_1  | 
elk_1  | ==> /var/log/logstash/logstash-plain.log <==
elk_1  | [2018-08-11T17:35:08,371][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/opt/logstash/data/queue"}
elk_1  | [2018-08-11T17:35:08,380][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/opt/logstash/data/dead_letter_queue"}
elk_1  | [2018-08-11T17:35:08,990][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
elk_1  | [2018-08-11T17:35:09,025][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"aa287931-643e-47ae-bd8e-f982c75b2105", :path=>"/opt/logstash/data/uuid"}
elk_1  | [2018-08-11T17:35:09,779][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.3.2"}
elk_1  | [2018-08-11T17:35:13,753][WARN ][logstash.outputs.elasticsearch] You are using a deprecated config setting "document_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Document types are being deprecated in Elasticsearch 6.0, and removed entirely in 7.0. You should avoid this feature If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"document_type", :plugin=><LogStash::Outputs::ElasticSearch hosts=>[//localhost], manage_template=>false, index=>"%{[@metadata][beat]}-%{+YYYY.MM.dd}", document_type=>"%{[@metadata][type]}", id=>"c4ee5abcf701afed0db36d4aa16c4fc10da6a92bbd615d837cccdf2f368b7802", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_21596240-07d7-4d2e-b4e5-bb68516e5a61", enable_metric=>true, charset=>"UTF-8">, workers=>1, template_name=>"logstash", template_overwrite=>false, doc_as_upsert=>false, script_type=>"inline", script_lang=>"painless", script_var_name=>"event", scripted_upsert=>false, retry_initial_interval=>2, retry_max_interval=>64, retry_on_conflict=>1, action=>"index", ssl_certificate_verification=>true, sniffing=>false, sniffing_delay=>5, timeout=>60, pool_max=>1000, pool_max_per_route=>100, resurrect_delay=>5, validate_after_inactivity=>10000, http_compression=>false>}
elk_1  | [2018-08-11T17:35:13,823][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>6, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
elk_1  | [2018-08-11T17:35:15,074][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
elk_1  | [2018-08-11T17:35:15,090][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
elk_1  | [2018-08-11T17:35:15,360][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
elk_1  | [2018-08-11T17:35:15,518][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
elk_1  | [2018-08-11T17:35:15,525][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
elk_1  | [2018-08-11T17:35:15,569][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost"]}
elk_1  | [2018-08-11T17:35:16,370][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
elk_1  | [2018-08-11T17:35:16,445][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x2c697fd4 run>"}
elk_1  | [2018-08-11T17:35:16,602][INFO ][org.logstash.beats.Server] Starting server on port: 5044
elk_1  | [2018-08-11T17:35:16,643][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
elk_1  | [2018-08-11T17:35:17,096][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
elk_1  | 
elk_1  | ==> /var/log/kibana/kibana5.log <==
elk_1  | {"type":"log","@timestamp":"2018-08-11T17:35:20Z","tags":["listening","info"],"pid":247,"message":"Server running at http://0.0.0.0:5601"}

Now, Kibana and Elasticsearch seem to be perfectly fine, while logstash isn't doing anything because when I type something in the terminal, I get no response.

Running ps aux in the container bash terminal, I get the following:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  21332  3592 ?        Ss   17:50   0:00 /bin/bash /usr/local/bin/start.sh
root        20  0.0  0.0  29272  2576 ?        Ss   17:50   0:00 /usr/sbin/cron
elastic+    86 18.0  4.4 5910168 1479108 ?     Sl   17:50   0:46 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -X
elastic+   112  0.0  0.0 135668  7328 ?        Sl   17:50   0:00 /opt/elasticsearch/modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller
logstash   226 43.6  2.2 5714032 726940 ?      SNl  17:50   1:47 /usr/bin/java -Xms1g -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djav
kibana     243 20.0  0.4 1315812 155744 ?      Sl   17:50   0:49 /opt/kibana/bin/../node/bin/node --max-old-space-size=250 --no-warnings /opt/kibana/bin/../src/cli -l /var/log/kibana/kibana5.log
root       245  0.0  0.0   7612   672 ?        S    17:50   0:00 tail -f /var/log/elasticsearch/elasticsearch.log /var/log/logstash/logstash-plain.log /var/log/kibana/kibana5.log
root       323  1.3  0.0  21488  3544 pts/0    Ss   17:54   0:00 bash
root       340  0.0  0.0  37656  3300 pts/0    R+   17:54   0:00 ps aux

Running ll /var/log/logstash/ in the container bash terminal, I get the following:

total 16
drwxr-xr-x 1 logstash logstash 4096 Aug 11 17:51 ./
drwxr-xr-x 1 root     root     4096 Jul 26 14:27 ../
-rw-r--r-- 1 root     root        0 Aug 11 17:50 logstash.err
-rw-r--r-- 1 logstash logstash 3873 Aug 11 17:51 logstash-plain.log
-rw-r--r-- 1 logstash logstash    0 Aug 11 17:51 logstash-slowlog-plain.log
-rw-r--r-- 1 root     root     3964 Aug 11 17:51 logstash.stdout

Now, I did change logstash.conf to have the following:

input { stdin {} }

output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
}

Still when I type something in the terminal, there is nothing in the discover section of Kibana, neither has any index pattern created...

Running ps aux in the container bash terminal, I get the following:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  21332  3600 ?        Ss   17:40   0:00 /bin/bash /usr/local/bin/start.sh
root        21  0.0  0.0  29272  2568 ?        Ss   17:40   0:00 /usr/sbin/cron
elastic+    87 12.0  4.5 5912216 1484068 ?     Sl   17:40   0:52 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -X
elastic+   113  0.0  0.0 135668  7332 ?        Sl   17:40   0:00 /opt/elasticsearch/modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller
logstash   224 27.8  2.3 5714032 771528 ?      SNl  17:40   1:58 /usr/bin/java -Xms1g -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djav
kibana     241 12.0  0.5 1322444 181228 ?      Sl   17:40   0:50 /opt/kibana/bin/../node/bin/node --max-old-space-size=250 --no-warnings /opt/kibana/bin/../src/cli -l /var/log/kibana/kibana5.log
root       246  0.0  0.0   7612   692 ?        S    17:40   0:00 tail -f /var/log/elasticsearch/elasticsearch.log /var/log/logstash/logstash-plain.log /var/log/kibana/kibana5.log
root       317  1.0  0.0  21488  3744 pts/0    Ss   17:47   0:00 bash
root       334  0.0  0.0  37656  3356 pts/0    R+   17:48   0:00 ps aux

Running ll /var/log/logstash/ in the container bash terminal, I get the following:

total 16
drwxr-xr-x 1 logstash logstash 4096 Aug 11 17:41 ./
drwxr-xr-x 1 root     root     4096 Jul 26 14:27 ../
-rw-r--r-- 1 root     root        0 Aug 11 17:40 logstash.err
-rw-r--r-- 1 logstash logstash 3873 Aug 11 17:41 logstash-plain.log
-rw-r--r-- 1 logstash logstash    0 Aug 11 17:41 logstash-slowlog-plain.log
-rw-r--r-- 1 root     root     3964 Aug 11 17:41 logstash.stdout

I have been spending a good amount of time with no luck here, so any help would be highly appreciated!


Solution

  • So, I did find a solution thanks to the owner of the elk image repo.

    I followed instructions from this page. That is, I entered the container bash by running docker exec -it <container-name> bash, and then (inside the container terminal) I ran the command /opt/logstash/bin/logstash --path.data /tmp/logstash/data -e 'input { stdin { } } output { elasticsearch { hosts => ["localhost"] } }'.

    The problem was that although the Logstash service had been started, it did not have an an interactive terminal. The command above, addresses this problem.

    The following logs were displayed inside the container terminal:

    Sending Logstash's logs to /opt/logstash/logs which is now configured via log4j2.properties
    [2018-08-12T06:28:28,941][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/tmp/logstash/data/queue"}
    [2018-08-12T06:28:28,948][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/tmp/logstash/data/dead_letter_queue"}
    [2018-08-12T06:28:29,592][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
    [2018-08-12T06:28:29,656][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"29cb946b-2bed-4390-b0cb-9aad6ef5a2a2", :path=>"/tmp/logstash/data/uuid"}
    [2018-08-12T06:28:30,634][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.3.2"}
    [2018-08-12T06:28:32,911][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>6, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
    [2018-08-12T06:28:33,646][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
    [2018-08-12T06:28:33,663][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
    [2018-08-12T06:28:34,107][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
    [2018-08-12T06:28:34,205][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
    [2018-08-12T06:28:34,212][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
    [2018-08-12T06:28:34,268][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost"]}
    [2018-08-12T06:28:34,364][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
    [2018-08-12T06:28:34,442][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
    [2018-08-12T06:28:34,496][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x5dcf75c7 run>"}
    [2018-08-12T06:28:34,602][INFO ][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/logstash
    The stdin plugin is now waiting for input:
    [2018-08-12T06:28:34,727][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
    [2018-08-12T06:28:35,607][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9601}
    

    And the following inside my server terminal:

    elk_1  | ==> /var/log/elasticsearch/elasticsearch.log <==
    elk_1  | [2018-08-12T06:28:34,777][INFO ][o.e.c.m.MetaDataIndexTemplateService] [jqTz2zS] adding template [logstash] for index patterns [logstash-*]
    elk_1  | [2018-08-12T06:28:35,214][INFO ][o.e.c.m.MetaDataCreateIndexService] [jqTz2zS] [logstash-2018.08.12] creating index, cause [auto(bulk api)], templates [logstash], shards [5]/[1], mappings [_default_]
    elk_1  | [2018-08-12T06:28:36,207][INFO ][o.e.c.m.MetaDataMappingService] [jqTz2zS] [logstash-2018.08.12/hiLssj14TMKd5lzBq6tvrw] create_mapping [doc]
    

    Doing so, an index pattern was indeed created inside Kibana and I started to receive messages inside the discover tab.