Observability

Amazon OpenSearch Service provides new capabilities to help solve your observability problems. Use open interfaces to collect, route, and transform telemetry data (including OpenTelemetry, Fluentd, Fluentbit, Logstash, Data Prepper, and more). You can search and analyze large amounts of semi-structured data with native capabilities. You can visualize, monitor, and alert with anomaly detection observability features of OpenSearch Dashboards, and conduct interactive analysis and visualization on data with Piped Processing Language (PPL), a query interface.

Amazon OpenSearch Service approaches the observability, trace analytics, log analytics, and application performance monitoring (APM) problem across four dimensions:

Collect: First, you need to collect data for analysis. Collection includes gathering, enriching, filtering, transforming, and normalizing data from multiple sources.

Detect: Often customers don’t detect issues as soon as they began, there is often a lag from when an issue starts and when you are notified. You want to reduce this as much as possible. Detection should be proactive and multi-faceted (i.e. alarms on telemetry). Anomaly detection is a key tool, as well as the ability to link together related alarms to reduce alarm fatigue. A core component of detection is also visualization and monitoring, which Amazon OpenSearch Service does with a component called OpenSearch Dashboards. You can even interactively analyze the data with tools like PPL.

Investigate: Investigation is where people spend the most amount of time during an operational event—and the investigation usually takes multiple people. This is the largest contributor to Mean Time to Incident (MTTI) and Mean Time to Recovery (MTTR). Cutting through the chaos and understanding what to focus on remains a difficult task. Leverage logs, metrics, and tracing to help you quickly conduct root cause analysis—while correlating across metrics, logs and traces. And collaborate on the investigations and document your analysis with OpenSearch Dashboard notebooks. 

Remediate: After you identify the cause of a failure, you need to remediate it. There is nothing worse than trying to fix something and making the situation worse. Don’t forget to do a post-event analysis to determine how you could have prevented the failure in the first place. Document proposed changes so you can prevent the issue from recurring. Your goal should be to ensure the same issue never happens again--but if it does, that you can identify and remediate it automatically.

For log analytics, there are several tasks to build an ingest pipe. Producers are back-end servers, AWS services, web servers, and more—including OpenTelemetry, AWS Distro for OpenTelemetry, Jaeger, and Zipkin. Collectors move the data from the source, possibly transforming the data locally. For AWS native services, you can use Amazon Kinesis Agent or Amazon CloudWatch Agent. For open source, common collectors are Elastic Beats, Fluentd, Fluentbit, or OpenTelemetry Collector. Aggregators buffer information from the collectors, which importantly reduces the overall connections to Amazon OpenSearch Service. Amazon OpenSearch Service then indexes and analyzes the result of the aggregators. To visualize and monitor the results, you can use OpenSearch Dashboards or Kibana.

Sometimes Application Performance Monitoring (APM) is the first maturity level of observability. But APM alone is not enough. Is your application actually performing as expected, even if your application monitoring dashboard is all green? Are your customers getting the user experience they need? What’s the usage of your application? Which parts of your application are hitting scale limits? From which geographic region are you seeing the biggest growth? Which trends can you visualize and plan for? If you could gather metrics, you could have confidence that when you deploy new code or change your infrastructure, you can see the impact of these changes. Observability advances APM to answer these additional questions.

Amazon OpenSearch Service’s Observability functionality allows you to go beyond simple monitoring to understand not just what events are happening, but why they are happening. In this workshop, learn how to instrument, collect, and analyze metrics, traces, and log data all the way from user front ends to service backends and everything in between. Put this together with Amazon OpenSearch Service, AWS Distro for OpenTelemetry, FluentBit, and Data Prepper.