Official logstash elastic cloud module
Official doc for starting with
My logstash.yml looks like:
cloud.id: "Test:testkey"
cloud.auth: "elastic:password"
With 2 spaces in front and no space at end, within ""
This is all I have in logstash.yml and nothing else,
And I am getting:
[2018-08-29T12:33:52,112][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://myserverurl:12345/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'https://myserverurl:12345/'"}
And the my_config_file_name.conf looks like:
input{jdbc{...jdbc here... This works, as I see data in windows console}}
output {
stdout { codec => json_lines }
elasticsearch {
hosts => ["myserverurl:12345"]
index => "my_index"
# document_id => "%{brand}"
}
What I am doing is hitting bin/logstash on windows cmd,
It loads data from database that I have configured in input of conf file and then shows me error, I want to index my data from MySQL to elasticsearch on Cloud, I took 14 days trial and created a test index, for learning purpose as I later have to deploy it.
My Pipeline looks like:
- pipeline.id: my_id
path.config: "./config/conf_file_name.conf"
pipeline.workers: 1
If logs won't include senistive data, I can also provide them.
Basically I wan't to sync (schedule check) my MYSQL data with ElasticSearch on cloud i.e. AWS
The output shall be:
elasticsearch {
hosts => ["https://yourhost:yourport/"]
user => "elastic"
password => "password"
# protocol => https
# port => "yourport"
index => "test_index"
# document_id => "%{table_id}"
# - represent comments
as stated at: Configuring logstash with elastic cloud docs
The document provided while deploying app does not provide config for jdbc, jdbc as well need user and password even if defined in settings file i.e. logstash.yml