Amazon Kinesis Auto Scaling
work-dharmendra Nov 1
This is npm module which scale amazon kinesis as per current traffic needs. This module continuously monitor traffic in kinesis stream and split and merge shards as needed. Amazon provides Kinesis to process streaming data. Each kinesis stream can have multiple shards and each shard can have traffic upto 1000 records per sec. E.g. if your application needs to handle 5000 req/sec then you need to have 5 shards. Since traffic on your application can varies a lot so does number of shards. If your application needs to handle 20000 req/sec at peak time then you need to have 20 shards but when at other time you may required only 5 shards. Each shard cost money. It is important to have enough shards to serve current traffic but we should close additional shards if there is no need to save money. Currently amazon doesn't provide any auto scaling of kinesis stream. This npm module fill that missing link so that your application always have enough shard to process.