Difinition and some features
So in today's lesson we are going to talk about amazon kinesis firehose and this service is used to load real time streams into data lakes warehouses and analytic services.
So amazon kinesis firehose is an extract transform and load (ETL) service and it's used to capture transform and deliver streaming data.
It's also a fully managed service for delivering real time streaming data.
Its destinations could be:
- Amazon S3
- Amazon RedShift
- Elastic Search
So a destination of the amazon kinesis firehose is the output of the service.
It's used to load new data in near real time which is like 60 seconds and you need to know that this is not a real time streaming service, it's a near real time.
Unlike the Amazon Kinesis data streams, which is a real time streaming service.
Also the delivery streams automatically scale up and down in order to handle gigabytes per second or more of input data rate.
So amazon kinesis firehose could be used to transform the data before being delivered to a specific destination.
Sources and Destinations of Amazon Kinesis Firehose.
Sources
So it could be a producer application writing its data directly to the kinesis firehose like:
- Amazon CloudWatch Logs
- Amazon CloudWatch Events
- AWS IOT
- Kinesis Data Stream
- Kinesis Agent
Note: Kinesis Agent is a prebuilt java application which monitors certain files and then sends this information to the delivery stream.
Destinations
- Amazon S3
- Amazon RedShift
- Elastic Search
- Splunk
- And many more...
Top comments (0)