logstashlogstash-jdbc

Have logstash JDBC input `last_run_metadata_path` on an external source?


We want to run logstash in an ECS environment where the instances have transient lives. They might be long lived, but they aren't permanent. Thus, having logstash store the sql_last_value in the last_run_metadata_path doesn't work if the instance is removed and then brought up again. That state gets lost.

Is there a way to have that value pulled from a S3 path? Or via a DB query? I'm just looking somewhere persistent that won't vanish when the instance gets deleted and brought up again.

last_run_metadata_path => "/home/ckessel/.logstash_source_a"  <-- need that to be S3 or some such

Solution

  • The file input just reads the saved value from a file. So the location can be anything that the Ruby IO::read function can read. That does not include s3 or a database. It needs to be a filesystem, but it can be network mounted.