Amazon CloudWatch Documentation
Amazon CloudWatch is a monitoring and management service designed to provide data and actionable insights for AWS, hybrid, and on-premises applications and infrastructure resources. With CloudWatch, you can collect and access your performance and operational data in the form of logs and metrics from a single platform.
CloudWatch is designed to give you actionable insights that help you optimize application performance, manage resource utilization, and understand system-wide operational health. CloudWatch provides visibility of metrics and logs data, data retention (metrics), and the ability to perform calculations on metrics. You can use CloudWatch Container Insights to monitor, troubleshoot, and alarm on your containerized applications and microservices.
Collect
CloudWatch Log classes
There are two log classes:
1. Amazon CloudWatch Logs Infrequent Access (Logs-IA).
2. Amazon CloudWatch Logs Standard.
Collect and store logs
The Amazon CloudWatch Logs service helps you to collect and store logs from your resources, applications, and services. There are three main categories of logs 1) Vended logs. These are natively published by AWS services on behalf of the customer. 2) Logs that are published by AWS services. 3) Custom logs. These are logs from your own application and on-premises resources.
Built-in metrics
Amazon CloudWatch helps you to collect default metrics from many AWS services.
Custom Metrics
Amazon CloudWatch helps you to collect custom metrics from your own applications to help you monitor operational performance, troubleshoot issues, and spot trends. You can use CloudWatch Agent or the PutMetricData API action to publish these metrics to CloudWatch.
Collect and aggregate container metrics and logs
Container Insights is designed to simplify the collection and aggregation of curated metrics and container ecosystem logs. CloudWatch custom metrics can be extracted from these ingested logs and further analyzed using CloudWatch Logs Insights’ advanced query language.
Collect and aggregate Lambda metrics and logs
CloudWatch Lambda Insights helps with the collection and aggregation of curated metrics and logs from AWS Lambda functions. CloudWatch custom metrics can be extracted from these ingested logs and further analyzed using CloudWatch Logs Insights’ advanced query language.
Stream Metrics
Amazon CloudWatch Metric Streams helps you to create continuous streams of metrics to a destination of your choice. Metrics Streams makes it easier to send CloudWatch metrics to popular third-party service providers using an Amazon Kinesis Data Firehose HTTP endpoint. You can also direct your metrics to your data lake on AWS.
Monitor
Cross-account observability across multiple AWS accounts
Cross-account observability in CloudWatch helps you monitor and troubleshoot applications that span multiple accounts within a Region. Cross-account observability in CloudWatch is designed to provide an interactive map of your cross-account applications using ServiceLens with drill downs to relevant metrics, logs, and traces.
Unified operational view with dashboards
Amazon CloudWatch dashboards enable you to create re-usable graphs and visualize your cloud resources and applications in a unified view. This is designed to give you system-wide visibility into operational health and the ability to troubleshoot issues.
Monitor metrics on AWS, on premises, and other clouds
Amazon CloudWatch supports querying from multiple data sources including your own custom data sources. Amazon CloudWatch multi data source querying allows you to set up your own data source using an AWS Lambda function.
Composite alarms
Amazon CloudWatch composite alarms enable you to combine multiple alarms and reduce alarm noise. If an application issue affects several resources in an application, you can set up receipt of a single alarm notification for the entire application instead of one for each affected service component or resource
High resolution alarms
Amazon CloudWatch alarms helps you to set a threshold on metrics and trigger an action. You can create high-resolution alarms, set a percentile as the statistic, and choose to specify an action or ignore one as you deem appropriate.
Logs and metrics correlation
Amazon CloudWatch is designed to correlate metrics and logs.
Application Insights
Amazon CloudWatch Application Insights helps provide setup of observability for your enterprise applications, to assist you in gaining visibility into the health of such applications. You can monitor this telemetry data to detect and correlate anomalies and errors, to notify you of any problems in your application.
Container monitoring insights
Container Insights provides dashboards in the CloudWatch console. These dashboards summarize the compute performance, errors, and alarms by cluster, pod/task, and service.
Database Insights
CloudWatch Database Insights consolidates logs and metrics from your applications, your databases, and the operating systems on which they run into a unified view in the console.
Internet Monitor
Internet Monitor is designed to provide visibility into how internet issues impact the performance and availability between your AWS-hosted applications and your end users. Internet Monitor helps you explore measurements for different timeframes and at different geographic granularities, and visualize the impact of issues. Internet Monitor is designed to deliver measurements to CloudWatch metrics and CloudWatch Logs, to support integrating health information for geographies and networks specific to your application. Internet Monitor is also designed to send health events to Amazon EventBridge, so you can set up notifications.
Lambda monitoring insights
Lambda Insights provides dashboards in the CloudWatch console. These dashboards are designed to summarize the compute performance and errors. Each dashboard includes the list of metrics for the selected time window and allows you to contextually dive deeper into application logs, AWS X-Ray traces, and performance events.
Anomaly Detection
Amazon CloudWatch Anomaly Detection applies machine-learning algorithms designed to assist you to analyze data of a metric and identify anomalous behavior.
ServiceLens
You can use Amazon CloudWatch ServiceLens to help you visualize and analyze the health, performance, and availability of your applications in a single place. CloudWatch ServiceLens ties together CloudWatch metrics and logs as well as traces from AWS X-Ray to help you obtain a complete view of your applications and their dependencies. . CloudWatch ServiceLens is designed to allow visibility into your applications in three main areas: Infrastructure monitoring (using metrics and logs to understand the resources supporting your applications), transaction monitoring (using traces to understand dependencies between your resources), and end user monitoring (using canaries to monitor your endpoints and notify you when your end user experience has degraded).
Synthetics
Amazon CloudWatch Synthetics helps you to monitor application endpoints. You can run tests on your endpoints and receive alerts if your application endpoints don’t behave as expected. You can also use CloudWatch Synthetics to isolate alarming application endpoints and map them back to underlying infrastructure issues. CloudWatch can collect canary traffic, which can assist you in verifying your customer experience enabling you to discover issues.
RUM
Amazon CloudWatch RUM is designed to give you visibility into your applications’ client-side performance. It is designed to allow you to collect client-side data on web application performance in near real time to identify and debug issues. CloudWatch RUM is designed to aggregate data on your users' journey through your application.
Act
Auto Scaling
Auto Scaling helps you automate capacity and resource planning. You can set a threshold to alarm on a key metric and trigger an Auto Scaling action.
Respond to operational changes with CloudWatch Events
CloudWatch Events is designed to provide a near real-time stream of system events that describe changes to your AWS resources.
Alarm and take action on EKS, ECS, and k8s clusters
For Amazon EKS and k8s clusters, CloudWatch Container Insights helps you to alarm on compute metrics to trigger auto scaling policies on your Amazon EC2 Auto Scaling group and provides you the ability to stop, terminate, reboot, and recover any Amazon EC2 instance.
Analyze
Analyze logs
Amazon CloudWatch Logs Insights enables you to query logs sent to CloudWatch in the AWS console, or write queries with aggregations, filters, and regular expressions. In addition, you can visualize time-series data, drill down into individual log events, and export query results to CloudWatch Dashboards.CloudWatch can recognize shared structures among log records, extract notable content and trends, and identify anomalies.
With CloudWatch Logs Live Tail, you can analyze streaming log data from a central view. You can launch contextual queries to transition from log monitoring to deeper log analytics and incident investigation and resolution.
Analyze metrics
Amazon CloudWatch Metrics Insights is a SQL-based query engine that enables you to identify trends and patterns within operational metrics with flexible querying and metric aggregations.
Granular data and extended retention
Amazon CloudWatch helps you to monitor trends and seasonality. With CloudWatch, you can also collect health metrics including custom ones, such as those coming from your on-premises applications.
Custom operations on metrics
Amazon CloudWatch Metric Math helps you to perform calculations across multiple metrics for real-time analysis. You can visualize these computed metrics in the AWS Management Console, add them to CloudWatch dashboards, or retrieve them using the GetMetricData API action. Metric Math supports arithmetic operations such as +, -, /, *, and mathematical functions such as Sum, Average, Min, Max, and Standard Deviation.
Analyze Lambda metrics, logs, and traces
Lambda Insights help simplify the analysis of observable data from metrics, logs, and traces by simplifying deep linking from automatic dashboards to granular performance events, application logs, and custom logs, using CloudWatch Logs Insights’ advanced query language.
Analyze container and Lambda telemetry
CloudWatch Container Insights and Lambda Insights helps you simplify the analysis of observable data from metrics, logs, and traces by simplifying deep linking from automatic dashboards to granular performance events, application logs, and custom logs using CloudWatch Logs Insights advanced query language.
Analyze time-series data
Amazon CloudWatch includes Contributor Insights, which enables you to analyze time-series data to obtain a view of the top contributors influencing system performance. Contributor Insights helps you understand who or what is impacting your system and application performance. You can create Contributor Insights rules to evaluate patterns in structured log events as they are sent to CloudWatch Logs, including logs from AWS services, custom logs sent by your service or on-premises servers, and other clouds. Contributor Insights is designed to evaluate these log events and display reports that show the top contributors and number of unique contributors in a dataset. A contributor is an aggregate metric based on dimensions contained as log fields in CloudWatch Logs. You can sort and filter contributor data based on your own custom criteria. Contributor Insights report data can be displayed on CloudWatch dashboards, graphed alongside CloudWatch metrics, and added to CloudWatch alarms.
Validate new features
Amazon CloudWatch Evidently allows you to validate new features across the full application stack before release. When launching new features, you can expose them to a small user base, monitor key metrics such as page load times or conversions, and then dial up traffic. Evidently also allows you to try different designs, collect user data, and release the most effective design in production.
Compliance and Security
Amazon CloudWatch is integrated with AWS Identity and Access Management (IAM) and designed to help you configure which users and resources have permission to access your data and how they can access it.
Data is encrypted at rest and during transfer. You can also use AWS KMS encryption to encrypt your log groups for added compliance and security.
Amazon CloudWatch Logs data protection is designed to help you to define data protection policies that can discover and protect sensitive data logged by systems and applications. This feature is designed to identify and mask sensitive information in your logs using ML and pattern matching based on the policy that you define. Data protection can help you streamline your architecture by offloading data protection logic from your applications. You can define your data protection policies to scan logs as they are ingested to determine how much sensitive data they contain and mask sensitive data that is detected. Masked data can also be unmasked for validation by security engineers through elevated privileges with IAM.
Additional Information
For additional information about service controls, security features and functionalities, including, as applicable, information about storing, retrieving, modifying, restricting, and deleting data, please see https://docs.aws.amazon.com/index.html. This additional information does not form part of the Documentation for purposes of the AWS Customer Agreement available at http://aws.amazon.com/agreement, or other agreement between you and AWS governing your use of AWS’s services.