AWS Cloud Operations & Migrations Blog

Category: Monitoring and observability

How to Monitor Databricks with Amazon CloudWatch

This post was written by Lei Pan and Sajith Appukuttan from Databricks. In this post, we look closely at monitoring and alerting systems – both critical components of any production-level environment. We’ll start with a review of the key reasons why engineers should build a monitoring/alerting system for their environment, the benefits, as well as […]

Read More

Deploy Multi-Account Amazon CloudWatch Dashboards

Organizations building modern applications require a way to gain actionable insights into their Amazon Elastic Compute Cloud (Amazon EC2) workloads. Amazon CloudWatch is a monitoring and observability service that collects operational data from logs, metrics, and events. The service lets customers monitor your resources spread across different accounts or regions in a single view, visualize […]

Read More

Viewing custom metrics from statsd with Amazon Managed Service for Prometheus and Amazon Managed Grafana

Monitoring applications based on custom metrics is important for a resilient system. One of the mechanisms to generate custom metrics from applications is statsd – a NodeJs process to collect custom application performance metrics periodically. However, statsd doesn’t provide long-term storage, rich querying, visualization, or an alerting solution. Amazon Managed Service for Prometheus and Amazon […]

Read More

Viewing collectd statistics with Amazon Managed Service for Prometheus and Amazon Managed Service for Grafana

Monitoring systems are essential for a resilient solution. A popular tool to monitor Linux-based physical or virtual machines is collectd – a daemon to collect system and application performance metrics periodically. However, collectd doesn’t provide long-term storage for metrics, rich querying, visualization, or an alerting solution. The Amazon Managed Service for Prometheus is a serverless […]

Read More

What is observability and Why does it matter? – Part 1

Before defining observability, consider the following example: You run an e-commerce site, and you’re interested in understanding the customer experience of the site, as well as how that translates into sales. You have identified that long page-loading times lead to poor customer experience, which in turn leads customers to abandon their carts and buy competing […]

Read More

How McAfee used Amazon CloudWatch to monitor a multi-PB data migration to Databricks on AWS

This blog post was contributed by Kanishk Mahajan@AWS; Hashem Raslan, Manager, Engineering@McAfee; Anastasia Zamyshlyaeva, Vice President, Data Engineering@McAfee McAfee, a global leader in online protection security enables home users and businesses to stay ahead of fileless attacks, viruses, malware, and other online threats. McAfee wanted to create a centralized data platform as a single source […]

Read More

Integrate Amazon CloudWatch metrics with ServiceNow using Amazon Managed Grafana

ServiceNow ITSM is a cloud-based platform designed to improve IT services, increase user satisfaction, and boost IT flexibility and agility. With ServiceNow IT Service Management, you can consolidate your legacy on-premise systems and IT tools into our single data model to transform the service experience, automate workflows, gain real-time visibility, and improve IT productivity. Amazon […]

Read More

Monitoring Amazon EMR on EKS with Amazon Managed Prometheus and Amazon Managed Grafana

Apache Spark is an open-source lightning-fast cluster computing framework built for distributed data processing. With the combination of Cloud, Spark delivers high performance for both batch and real-time data processing at a petabyte scale. Spark on Kubernetes is supported from Spark 2.3 onwards, and it gained a lot of traction among enterprises for high performance and […]

Read More

Quantify custom application metrics with Amazon CloudWatch Logs and metric filters

Customers have valuable metrics emitted to their logs. Examples include web server response times, slow queries, purchases by partners, custom application metrics, and cache hits or misses. This data has unrealized potential value for increasing observability. Consumed by Amazon CloudWatch Logs and extracted using metric filters, customers can translate this data into actual CloudWatch metrics, […]

Read More

Automate time series network visualizations for AWS PrivateLink using Amazon CloudWatch Contributor Insights

AWS PrivateLink is a highly available, scalable technology that lets you connect your Amazon Virtual Private Cloud (VPC) to supported AWS services without requiring public internet traversal. It also lets you privately connect to services hosted by other AWS accounts (VPC endpoint services) and supported AWS Marketplace partner services. Amazon CloudWatch Contributor Insights is a […]

Read More