AWS Lambda launches checkpointing for Amazon Kinesis and Amazon DynamoDB Streams
AWS Lambda now allows customers to automatically checkpoint records that have been successfully processed for Amazon Kinesis and Amazon DynamoDB Streams, using a new parameter, FunctionResponseType. When customers set this parameter to “Report Batch Item Failure”, if a batch fails to process, only records after the last successful message are retried. This reduces duplicate processing, and gives customers more options for failure handling.
Customers can use this checkpointing in addition to other failure handling features already available for Amazon Kinesis and Amazon DynamoDB Streams. If a failure occurs, Lambda prioritizes checkpointing, if enabled, over other mechanisms to minimize duplicate processing. Today, when customers use failure handling features such as BisectBatchOnError, they may incur duplicate processing. With checkpointing enabled, Lambda will only process the records that failed to process the first time, reducing duplicate records.
To get started, customers can enable checkpointing for Kinesis or DynamoDB Streams by setting the FunctionReponseType parameter to the value ReportBatchItemFailure on the AWS Management Console, AWS CLI, AWS SAM, or AWS SDK for Lambda. This feature requires no additional charge to use. This feature is available in all AWS Regions where Lambda, Kinesis, and DynamoDB are available. To learn more about setting up an analytics workload on Lambda, read the Lambda Developer Guide (Kinesis, DynamoDB) and the AWS Compute Blog.