How do I make my Lambda function idempotent to prevent inconsistencies and data loss in my application?
Last updated: 2019-05-28
I've heard that I should make my AWS Lambda function code idempotent. How do I do that?
In programming, idempotency is the capacity of an application or component to identify repeated events and respond accordingly to prevent duplicated, inconsistent, or lost data.
Making your AWS Lambda function idempotent requires designing your function logic to treat duplicated events correctly. Idempotent function logic is more likely to reduce:
- Unnecessary API calls
- Code processing time
- Data inconsistency
To make your function idempotent, its code must properly validate input events and identify if the events have been processed before. How your application works dictates the best way to do this. For guidance, see the example function logic and best practices below.
Example function logic
This example Lambda function logic applies to most cases:
- Extract the value of a unique attribute of the input event. (For example, a transaction or purchase ID.)
- Check if the attribute value exists in a control database (such as an Amazon DynamoDB table).
Note: Adding AWS services to your architecture might incur additional costs. For more information, see Amazon DynamoDB pricing or AWS Pricing.
- If a unique value exists (indicating a duplicated event), gracefully terminate the execution (that is, without throwing an error). If a unique value doesn't exist, continue the execution normally.
- When the function work finishes successfully, include a record in the control database.
- Finish the execution.
Keep these best practices in mind as you design your Lambda function for idempotency:
- Plan idempotency features before developing your application.
- Make sure that your Lambda code terminates gracefully when it processes a duplicate event. Throwing errors can cause further retries, either by Lambda or by another service that invokes your function.
- Change the Lambda function timeout setting if needed so that the full execution is handled correctly. Using a separate service to persist data and control duplicated events might require API calls to HTTPS endpoints. This might then require more time than the default three seconds.
- Test and optimize your function as much as possible, simulating a real scenario and rate of requests. This is critical in helping to prevent the idempotency logic from causing side effects such as timeouts, excessive latency, or bottlenecks.
- Store session data using a service that can be easily escalated and that provides high throughput, such as DynamoDB.