What’s new in AWS Observability at re:Invent 2023

Let’s recap the week at AWS re:Invent 2023 with a round-up of the AWS Observability launches across Amazon CloudWatch, Amazon Managed Grafana, and Amazon Managed Service for Prometheus. From automatic instrumentation and operation of applications in CloudWatch, to agentless scraping of Prometheus metrics in Managed Service for Prometheus, read on to learn about the features we showcased at re:Invent. You can also watch the What’s new with AWS observability and operations breakout session for demos of these capabilities.

Amazon CloudWatch Application Signals (Preview)
Amazon CloudWatch Application Signals (Preview) makes it easy to automatically instrument and operate applications on AWS. You can track application performance against your most important business objectives without the undifferentiated heavy lifting of manual instrumentation, metrics computations, and correlating observed problem to root cause. CloudWatch Application Signals provides standardized metrics such as volume, latency, and errors for each of your applications with pre-built dashboards. In as few as three clicks, you can spot anomalies, drill into the most important metrics, and identify the root cause of issues with correlated metrics logs and traces. Read the AWS News Blog for more details and watch the Application monitoring for modern workloads breakout.

Amazon CloudWatch Logs Anomaly Detection
Powered by AI/ML, Amazon CloudWatch Logs Anomaly Detection is an automated logs analytics feature that helps you cluster related logs to accelerate log investigation, compare changes to your logs over time and find what changed, and monitor your logs and notify you when an unusual behavior occurs for faster remediation. This capability helps you quickly sift through tens of thousands of complex log entries to identify the root cause of issues when they occur. Learn more in the AWS News Blog and check out the breakout session Accelerate insights using Amazon CloudWatch Logs ML-powered analytics.

Amazon CloudWatch Logs Infrequent Access
Amazon CloudWatch Logs Infrequent Access (Logs IA) is a new log class for cost-effectively consolidating all your logs natively on AWS, and helps you to improve visibility into your overall application health. CloudWatch Logs IA offers a subset of CloudWatch Logs’ capabilities including managed ingestion, cross-account log analytics, and encryption with a lower per GB ingestion price making Logs IA ideal for ad-hoc querying and after-the-fact forensic analysis on infrequently accessed logs. Read the AWS News Blog and watch the Get actionable insights from Amazon CloudWatch Logs breakout for more details.

Amazon CloudWatch natural language query generation (Preview)
Easily and quickly generate queries for your logs and metrics data using plain language with Amazon CloudWatch natural language query generation powered by generative AI for Logs Insights and Metrics Insights (Preview). By simplifying the query generation process, you can accelerate insights from your observability data without needing extensive knowledge of the query language. Learn more in the AWS News Blog. and watch a demo of this in the What’s new with AWS observability and operations breakout.

Amazon CloudWatch multi data source querying
Gain visibility across your hybrid and multicloud metrics in a single view with Amazon CloudWatch multi data source querying. With this feature, you can consolidate and visualize metrics from sources such as Amazon OpenSearch Service, Amazon Managed Service for Prometheus, Azure Monitor, your own custom data sources, and query those metrics in real time, increasing visibility into your application health and helping you resolve critical events faster. Read the AWS News Blog and watch this demo in the What’s new with AWS observability and operations breakout.

Amazon CloudWatch out-of-the-box best practice alarms
Get out-of-the box, best practice alarm recommendations for AWS service-vended metrics that will help you quickly set up essential monitoring for your AWS resources. See metric descriptions of AWS resources in dashboards across the AWS console for faster troubleshooting. Learn more here. Or check out the demo in the What’s new with AWS observability and operations breakout.

Amazon CloudWatch Container Insights (now available with enhanced observability for EKS)
Amazon CloudWatch Container Insights now delivers enhanced observability for Amazon Elastic Kubernetes Service (EKS) with out-of-the-box detailed health and performance metrics, including container-level EKS performance metrics, Kube-state metrics, and EKS control plane metrics for faster problem isolation and troubleshooting. Enhanced observability enables you to visually drill up and down across various container layers and easily spot issues like memory leaks in individual containers, reducing mean time to resolution. Learn more in this blog and watch this breakout Best practices for container observability to see it in action.

Amazon Managed Service for Prometheus collector
Amazon Managed Service for Prometheus collector is a new feature for automatic discovery and monitoring of Prometheus metrics for Amazon EKS applications and infrastructure. Prometheus is a popular open-source tool for monitoring and alerting that uses a pull-based collection process called scraping to discover, collect, and filter metrics. With the collector, Amazon EKS customers can now auto-discover and monitor their applications without having to manage, secure, scale, or reliably operate any Prometheus metric collection scrapers. Learn more in this blog and check out the Best practices for container observability breakout session.

Amazon Managed Grafana community plugins
Extend your Amazon Managed Grafana visualization experience with Grafana community plugins of your choice. You can discover and install Grafana community plugins directly from your workspace. Plugins enable you to extend your Grafana experience, unifying data from a wider variety of data sources with visualizations tailored to analyze your unique datasets. Learn more in this blog.

Resources

One Observability Workshop
The One Observability Workshop provides a broad hands-on experience on AWS services that help you monitor and gain insights your application performance and health. You’ll learn about logging, metrics, container monitoring and tracing techniques.

AWS Observability Best Practices
This Get Started guide includes best practices for observability: What do to, what not to do, and a collection of recipes on how to do them. Most of the guide is vendor agnostic and represents what any good observability solution will provide.

AWS CDK Observability Accelerator
The AWS CDK Observability Accelerator for Amazon EKS is a set of opinionated modules to help you set up observability for your AWS environments with AWS Native services and AWS-managed observability services such as Amazon Managed Service for Prometheus, Amazon Managed Grafana, AWS Distro for OpenTelemetry (ADOT) and Amazon CloudWatch.

AWS Observability Training Course
This Training & Certification four-part course introduces you to the fundamentals of AWS Observability and the services that can help elevate your cloud operations practice.

Author