AWS Distro for OpenTelemetry now available for public preview

Today’s distributed applications and systems are complex and constantly changing, making system observability challenging. For example, customers use multiple AWS SDKs and agents from different monitoring services to collect and analyze different performance data for their applications. Yesterday we announced the AWS Distro for OpenTelemetry, a 100% open source distribution of the OpenTelemetry project, which makes it easier for developers to collect and send application metrics and traces to multiple AWS monitoring services. The OpenTelemetry project provides a single set of open source APIs, SDK libraries, and agents for capturing metrics and distributed traces from applications. In our collaboration with the OpenTelemetry community, we contribute all changes back to the upstream project.

Open Telemetry offers a community-driven way to standardize the collection and transfer of telemetry data. AWS Distro for OpenTelemetry aims to make this even easier for developers to use OpenTelemetry components to instrument their application once. They can then send correlated performance data to AWS monitoring services, including Amazon CloudWatch and AWS X-Ray for monitoring and analysis. AWS Distro for OpenTelemetry provides auto-instrumentation agents that automatically collect trace data from the applications, while lessening the need to do this through manual coding. Customers can collect metadata about the underlying infrastructure, such as an AWS Lambda function identifier or container identifier, and correlate this data with application performance data, providing insights for troubleshooting and resolving issues. The distro supports applications running on:

With the launch of AWS Distro for OpenTelemetry, AWS will continue to help drive advances in observability technologies, enhancing innovation and scalability for the entire OpenTelemetry community by contributing 100% of all changes to the upstream. For example, AWS has added functionality to upstream observability components, such as the OpenTelemetry language SDKs, collectors, and agents. We’ve added Java and JavaScript support for Amazon monitoring tools, such as CloudWatch and AWS X-Ray. We are also adding features that generally benefit the OpenTelemetry community at large, including the addition of the C++ Metrics API and SDK to implement the OpenTelemetry specification. We’re also working on a prototype Log API and SDK for C++ in collaboration with project maintainers. The prototype is building a model to help define and develop OpenTelemetry’s log specifications. This will benefit the entire community independently of any specific vendor observability platforms.

AWS shares the core values of the upstream OpenTelemetry project, including reducing the costs of software telemetry instrumentation and ensuring the availability of open source, high-quality technologies that everyone can use. In this we are grateful to be collaborating with all other OpenTelemetry stakeholders, including Microsoft, Google, LightStep, New Relic, Red Hat, and Splunk.

Components of AWS Distro for OpenTelemetry

The AWS Distro for OpenTelemetry consists of SDKs, auto-instrumentation agents, collectors, and exporters to send data to backend services. AWS applies a comprehensive testing framework before each release to ensure that any overhead of OpenTelemetry components is quantifiable and predictable.

OpenTelemetry SDK: AWS enhanced upstream OpenTelemetry SDKs in three areas. First, a collection of AWS resource-specific metadata elements, including container identifier, EC2 instance identifier, Task, and Pod identifier are now available. Second, OpenTelemetry SDKs now support the X-Ray trace format and context. Third, OpenTelemetry SDKs now correlate ingested trace and metrics data from X-Ray and CloudWatch.
Auto-instrumentation agent: The Java auto-instrumentation agent in OpenTelemetry now supports AWS SDK and X-Ray trace data.
OpenTelemetry Collector: AWS-specific exporters have been added to the Collector to send traces to X-Ray and metrics to CloudWatch.

Getting started

To get started, you can deploy the Collector provided by the AWS Distro for OpenTelemetry on a host or container. You can use AWS CloudFormation templates when using Amazon ECS, and kubectl commands when using Amazon EKS to deploy the Collector as a sidecar or DaemonSet. To gain additional data from your applications, you can instrument them in Java and JavaScript using OpenTelemetry APIs, and use the Java auto-instrumentation agent.

AWS Distro for OpenTelemetry works alongside the CloudWatch agent to collect data needed for application monitoring. This means you can use the CloudWatch agent if it is already instrumented for your needs. You can use the CloudWatch agent to collect logs and infrastructure metrics. You can use OpenTelemetry to collect traces and application metrics. If you are using X-Ray currently, you can continue using the X-Ray SDK and daemon/agent for applications already instrumented. We recommend using AWS Distro if your objective is to standardize with the open source APIs from upstream OpenTelemetry.

We welcome you to download the public preview of the distribution. Please ask any questions you may have at our developer portal. You can reach us on Gitter to provide feedback.