Amazon Kinesis Services Comparison
| Feature | Kinesis Data Streams (KDS) | Kinesis Data Analytics (KDA) | Kinesis Data Firehose (KDF) | 
|---|---|---|---|
| Purpose | Real-time streaming ingestion and buffering | Real-time stream processing and analytics | Streaming ingestion → automatic delivery to destinations | 
| Data storage | Up to 7 days (configurable) | Temporary processing only | No storage — streams processed and delivered immediately | 
| Input | Producers (IoT devices, APIs, applications) | KDS or Firehose streams | Producers or KDA outputs | 
| Output | Multiple consumers (Lambda, KDA, Firehose, etc.) | Processed data → storage or streams | Amazon S3, Redshift, Elasticsearch, Splunk, custom HTTP endpoints | 
| Processing | None built-in (just streaming) | SQL queries or Apache Flink for real-time analytics | Optional transformation via Lambda | 
| Scalability | Very high throughput | Limited by analytics job capacity | Managed — auto scales | 
| Use cases | Real-time ingestion for processing later | Real-time analytics and transformation | Simplified data delivery to storage or analytics services | 
| Best for | Streaming large amounts of raw data that will be processed later | Real-time analytics on streaming data | Streaming ETL and delivery to analytics/storage without complex setup | 
How They Work Together
Here’s the flow:
- Kinesis Data Streams → collects raw streaming data from sources (e.g., IoT devices, API Gateway).
- Kinesis Data Analytics → reads from the stream to process, filter, or aggregate data in real time.
- Kinesis Data Firehose → takes either the raw stream (from KDS) or processed output (from KDA) and delivers it to storage destinations like Amazon S3, Redshift, or Elasticsearch.
Example — Bicycle Tracking System
| Service | Role in System | 
|---|---|
| Kinesis Data Streams | Collects raw location pings from bikes in real time during peak hours | 
| Kinesis Data Analytics | Processes location data to detect congestion, anomalies, or patterns in real time | 
| Kinesis Data Firehose | Delivers processed results to S3 for analytics and to DynamoDB for API access | 
Key Differences Summarized
- KDS = raw ingestion & buffering
- KDA = real-time analytics
- KDF = streaming ETL & delivery to storage
Example Bicycle Tracking Analogy
Kinesis Data Streams = water pipes carrying live location data from bikes into the city’s processing system.
Kinesis Data Analytics = treatment plant filtering and analyzing location streams — figuring out traffic jams or fastest routes.
Kinesis Data Firehose = distribution system sending the analyzed location data into storage lakes (S3) or dashboards for reporting.
 

 
    
Top comments (0)