Amazon MSK now offers consumer lag metrics and select topic-level metrics for free
Amazon Managed Streaming for Apache Kafka (MSK) now offers consumer lag metrics for new Amazon MSK clusters by default, making it easier for you to track whether your applications are consuming the latest data available in your Apache Kafka cluster. Consumer lag metrics quantify the difference between the latest data written to Apache Kafka topics in Amazon MSK and the data read by your applications. Monitoring consumer lag metrics allows application developers to identify slow or stuck consumers that are not keeping up with the latest data available in an Apache Kafka topic so they can take remedial actions such as scaling or rebooting those consumers.
Consumer lag metrics can be monitored using Amazon CloudWatch or Open Monitoring for Prometheus. Additionally, Amazon MSK has moved ten broker level and two topic level metrics into its default monitoring tier for all clusters, which allows you to monitor key cluster metrics such as bytes-in and bytes-out for free.
Consumer lag metrics are available for all new Amazon MSK clusters in all AWS regions where the service is currently available. For existing clusters, reach out to AWS Support to ask for enablement of consumer lag metrics. To learn more about monitoring your Amazon MSK clusters, see our documentation.