elasticsearchlogstashtwitter-streaming-api

Streaming API with logstash

I'm trying to use logstash exec plugin to ingest the twitter stream into elastic. As the API endpoint gives a stream of tweets(decahose), exec plugin doesn't seem to be working. Are there any better ways to do with logstash?

Solution

  • I just found that 'pipe' input plugin works fine than the 'exec' plugin in logstash

     input {
  pipe {
    command => 'curl --compressed -v -u <<username>>:<<password>> "https://gnip-stream.twitter.com/stream/sample10/accounts/<<accountName>/publishers/twitter/<<env>>.json"'
    codec => "json"
  }
}