Why is my Lambda IteratorAge metric increasing, and how do I decrease it?

Last updated: 2021-06-15

I see an increase (or spikes) in my AWS Lambda function's IteratorAge metric. Why is my function's iterator age increasing, and how do I decrease it?

Short description

A Lambda function's iterator age increases when the function can't efficiently process the data that's written to the streams that invoke the function. To decrease your function's IteratorAge metric, you must increase your stream processing throughput.

The following factors influence a function's IteratorAge metric:

Review this article to understand how each factor affects iterator age. Then, reconfigure your function or data stream to decrease your function's iterator age based on your use case. For more information, see Working with AWS Lambda function metrics.

Resolution

Decrease your function's runtime duration

A high runtime duration increases a function's iterator age. Decreasing the duration increases a function's throughput, which decreases a function's iterator age.

To decrease your function's runtime duration, do one or both of the following:

1.    Increase the amount of memory allocated to the function.

2.    Optimize your function code so that it takes less time to process records.

Note: For more information, see AWS Lambda and AWS X-Ray and Instrumenting AWS Lambda functions.

Increase your stream's shard count

A low number of shards in a stream increases a function's iterator age. Increasing the number of shards in a stream increases a function's throughput, which decreases a function's iterator age.

To increase your stream's shard count, follow the instructions in Resharding a stream.

Note: Shard splitting doesn't have an immediate effect on a function's iterator age. Existing records remain in the shards they were written to. Those shards must catch up on their backlog before the iterator age for the shards decreases. For more information, see Working with streams.

Increase your stream's batch size

If your function's runtime duration is independent from the number of records in an event, then increasing your function's batch size decreases the function's iterator age.

To increase your function's batch size, follow the instructions in Configuring a stream as an event source.

Note: If your function's duration is dependent on the number of records in an event, increasing your function's batch size doesn't decrease the function's iterator age. For more information, see Working with streams.

Make sure that your function gracefully handles invocation errors

Invocation errors can cause a Lambda function to take longer to process an event, or process the same event more than once. Because event records are read sequentially, a function can't progress to later records if a batch of records causes an error each time that it's retried. In this situation, the iterator age increases linearly as those records age.

It's important that your function gracefully handles records written to the stream. As you develop your function, logging and instrumenting your code can help you diagnose errors.

For more information, see the following:

Increase your stream processing throughput by testing different parallelization factor settings and using enhanced fan-out

To test parallelization factor settings

You can improve stream processing by configuring your function's parallelization factor to increase the number of concurrent Lambda invocations for each shard of a stream. You can also specify the number of concurrent batches that Lambda polls from each shard.

For example, if your parallelization factor is set to 10, you can have up to 50 concurrent Lambda invocations to process five Kinesis data shards.

For more information, see Using AWS Lambda with Amazon Kinesis and New AWS Lambda scaling controls for Kinesis and Amazon DynamoDB event sources.

Note: When increasing the number of concurrent batches per shard, Lambda manages in-order processing at the partition-key level.

To use enhanced fan-out

You can reduce latency and increase read throughput, by creating a data stream consumer with enhanced fan-out. Stream consumers get a dedicated connection to each shard that doesn't impact other applications that are also reading from the stream.

For more information, see Developing custom consumers with dedicated throughput (enhanced fan-out) and Using AWS Lambda with Amazon Kinesis.

Note: It's a best practice to use enhanced fan-out if you have many applications reading the same data, or if you're reprocessing streams with large records.