DEV Community

2

Amazon Kinesis Firehose

Difinition and some features

So in today's lesson we are going to talk about amazon kinesis firehose and this service is used to load real time streams into data lakes warehouses and analytic services.

So amazon kinesis firehose is an extract transform and load (ETL) service and it's used to capture transform and deliver streaming data.

It's also a fully managed service for delivering real time streaming data.
Its destinations could be:

  • Amazon S3
  • Amazon RedShift
  • Elastic Search

So a destination of the amazon kinesis firehose is the output of the service.

It's used to load new data in near real time which is like 60 seconds and you need to know that this is not a real time streaming service, it's a near real time.
Unlike the Amazon Kinesis data streams, which is a real time streaming service.

Also the delivery streams automatically scale up and down in order to handle gigabytes per second or more of input data rate.

So amazon kinesis firehose could be used to transform the data before being delivered to a specific destination.

Sources and Destinations of Amazon Kinesis Firehose.

Sources
So it could be a producer application writing its data directly to the kinesis firehose like:

  • Amazon CloudWatch Logs
  • Amazon CloudWatch Events
  • AWS IOT
  • Kinesis Data Stream
  • Kinesis Agent

Note: Kinesis Agent is a prebuilt java application which monitors certain files and then sends this information to the delivery stream.

Destinations

  • Amazon S3
  • Amazon RedShift
  • Elastic Search
  • Splunk
  • And many more...

GitHub
LinkedIn
Facebook
Medium

Image of Datadog

Create and maintain end-to-end frontend tests

Learn best practices on creating frontend tests, testing on-premise apps, integrating tests into your CI/CD pipeline, and using Datadog’s testing tunnel.

Download The Guide

Top comments (0)

Best Practices for Running  Container WordPress on AWS (ECS, EFS, RDS, ELB) using CDK cover image

Best Practices for Running Container WordPress on AWS (ECS, EFS, RDS, ELB) using CDK

This post discusses the process of migrating a growing WordPress eShop business to AWS using AWS CDK for an easily scalable, high availability architecture. The detailed structure encompasses several pillars: Compute, Storage, Database, Cache, CDN, DNS, Security, and Backup.

Read full post

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay