How do I make my Lambda function idempotent?
Last updated: 2021-04-14
I've heard that I should make my AWS Lambda function code idempotent to prevent inconsistencies and data loss in my application. How do I do that?
In programming, idempotency refers to the capacity of an application or component to identify repeated events and prevent duplicated, inconsistent, or lost data. Making your AWS Lambda function idempotent requires designing your function logic to treat duplicated events correctly.
Idempotent function logic can help to reduce the following issues:
- Unnecessary API calls
- Code processing time
- Data inconsistency
To make your function idempotent, the function's code must properly validate input events and identify if the events were processed before. How your application works dictates the best way to write the code.
Use the following example function logic and best practices for guidance.
Idempotent Lambda function logic example
Note: The following example Lambda function logic applies to most use cases.
1. Extract the value of a unique attribute of the input event. (For example, a transaction or purchase ID.)
2. Check if the attribute value exists in a control database. (For example, an Amazon DynamoDB table.) Then, depending upon the results, do the following:
If a unique value exists, end the action without producing an error.
If a unique value doesn't exist, proceed with the action that you originally designed.
3. When the function work finishes, include a record in the control database.
4. Finish the action.
Lambda function idempotency best practices
- Plan idempotency features before developing your application.
- Make sure that your Lambda code ends without producing an error when it processes a duplicate event.
Note: Throwing errors can cause further retries, either by Lambda or by another service that invokes your function.
- Change the Lambda function timeout setting, if needed, so that the full runtime is handled correctly.
Note: Using a separate service to persist data and control duplicated events might require API calls to HTTPS endpoints. API calls to HTTPS endpoints might then require more runtime than the default three seconds.
- Test and optimize your function as much as possible, simulating a real scenario and rate of requests.
Note: Testing and optimizing idempotent function logic is critical in helping to prevent potential side effects such as timeouts, excessive latency, or bottlenecks.
- Store session data using a service that can be easily escalated and that provides high throughput, such as DynamoDB.
Note: When making API calls to Amazon Elastic Compute Cloud (Amazon EC2), the service offers the parameter clientToken. The clientToken can be used to ensure that any mutating API request will run once only if the API request is retried.