Posted On: Nov 22, 2021

AWS Lambda has launched a new metric, OffsetLag, to monitor the performance of Amazon MSK, Self-managed Kafka, AmazonMQ, and RabbitMQ message queueing services. Up until now, Lambda users did not have visibility into how polling runs and had to increasingly rely on the Lambda support team to resolve delays in processing, leading to inefficiencies in data streaming. The OffsetLag metric is a measure of the total number of messages waiting in the message queue to be sent to the target Lambda function. This metric will provide transparency into the amount of data congestion in a message queue. Thus, developers can monitor the performance of events, set alarms and thresholds to check for undesirable congestion and quickly diagnose and solve inefficiencies in their data stream.

Basic insights of OffsetLag are available on CloudWatch for free. With these insights, you can study the consolidated number of messages lagging in all your event sources. Additionally, OffsetLag has the capability of gathering advanced insights into the health of the data stream at an event source mapping (ESM) level, or at a topic and/or partition level for Self-Managed Kafka. This is useful for any type of root-cause analysis such as determining which event types may be congesting a data stream. To gain access to these additional dimensions available in OffsetLag, check the pricing.

To learn more about OffsetLag and how it is calculated check the Lambda Developer Guide.