Background
i am studying about AWS kinesis,API gateway.
I understand that ,whenever requests hit API gateway,i can forward the data to a stream or i can choose to trigger a lambda(which will do some processing ).
Thoughts and Query
So,my thought was,if i can directly ,trigger a lambda from API gateway(When requests arrive,it is realtime),what is the advantage of having a kinesis stream(for realtime data processing)?
I could remove the streams and directly trigger lambda from API gateway(even create multiple APIs for different tasks)
Any thoughts in this scenario!
The solution you use really depends on the data you are processing and what you'd like to do with it. More information on the data and results of your scenario could narrow down a good AWS fit. Here's a simplification of 3 options:
Kinesis Streams basically supplies a time-shifting window into a large amount of data. It takes care of storing that data long enough so you can cherry pick relevant data or perform aggregations. Your analysis of the data could be stored in a database. Kinesis Streams is a good choice when storing all of the data coming in is unnecessary and would be costly.
Kinesis Firehose provides an endpoint for you to send your data to S3, Redshift, or Elastic Search (or some combination). You can then perform your analysis on that stored data. This is a good choice if you just want your raw data to end up in a database for later processing. But, you need to pay for the storage of that data. If you only need a subset of the data or the analysis results, it would be costly.
API Gateway to Lambda allows you to process the data in real-time. The lambda could do anything you want with the data, you have the most flexibility with this solution. But you have to process each request individually, while Kinesis Streams allows you to analyze a batch of data.