Posted On: Nov 25, 2019
AWS Lambda now supports Parallelization Factor, a feature that allows you to process one shard of a Kinesis or DynamoDB data stream with more than one Lambda invocation simultaneously. This new feature allows you to build more agile stream processing applications on volatile data traffic.
By default, Lambda invokes a function with one batch of data records from one shard at a time. For a single event source mapping, the maximum number of concurrent Lambda invocations is equal to the number of Kinesis or DynamoDB shards.
Now you can specify the number of concurrent batches that Lambda polls from a shard via a Parallelization Factor from 1 (default) to 10. For example, when Parallelization Factor is set to 2, you can have 200 concurrent Lambda invocations at maximum to process 100 Kinesis data shards. This helps scale up the processing throughput when the data volume is volatile and the IteratorAge is high.
You can get started with this feature with the AWS Management Console, AWS CLI, AWS CloudFormation, or AWS SDK for Lambda. It can be used at no additional cost in all AWS Regions where AWS Lambda is available. For more information on where AWS Lambda is available, see the AWS Region table. To learn more, see Using AWS Lambda with Amazon Kinesis and Using AWS Lambda with Amazon DynamoDB in the AWS Lambda Developer Guide, and New AWS Lambda scaling controls for Kinesis and DynamoDB event sources.