AWS Open Source Blog

AWS re:Invent is a learning conference hosted by Amazon Web Services (AWS) for the global cloud computing community. Open source is a huge part of AWS and it is also a huge part of re:Invent. We can’t wait to show you the amazing open source work that is happening and to share the latest from […]

AWS Distro for OpenTelemetry 0.14 is now available. You can download the latest AWS Distro for OpenTelemetry Collector image from the  Amazon Elastic Container Registry (Amazon ECR) Public Gallery. Release highlights OpenTelemetry Collector v0.38.0 Updated Lambda layers for Java, Java auto-instrumentation agent, JavaScript, and .Net New OpenTelemetry Operator image added to public Amazon ECR Added […]

This post was written by Dzenan Softic and Sam Dengler. Many organizations rely on Apache Airflow, an open source project, to orchestrate their data pipelines. In 2020, Amazon Web Services (AWS) released Amazon Managed Workflows for Apache Airflow (Amazon MWAA), which lets engineers focus on business solutions rather than on running and maintaining infrastructure for […]

Customers want better insight into understanding the behavior of their systems, but not all customers can afford to make significant code changes in their existing pipelines to add more observability. In this walkthrough, we explain how to get telemetry data from AWS Lambda Python functions, without having to change a line of code. Find the […]

This post was contributed by David Boeke, Bob Tordella, Jon Udell, and Nathan Wallace. Steampipe is an open source tool for querying cloud APIs in a universal way and reasoning about the data in SQL. To enable you to select * from cloud, the tool embeds Postgres, and it maps cloud APIs to database tables […]

DevOps is a common denominator for software delivery across industries. No matter the software, developers must ensure that infrastructure resources are provisioned; testing and delivery mechanisms are in place; and security, reliability, and scalability requirements are provided. That is why choosing the right DevOps tooling is central to a delivery team’s best practices, particularly in […]

In this post, Albert Choi, an intern on the Amazon Managed Service for Prometheus team, shares his experience of designing and implementing parallel compactors inside of the Cortex open source project. The addition to the compactors enables Cortex to handle large volumes of active metrics per tenant. This blog post details the work done as […]

This post was written by Kevin Bell and Stefan Prodan. Canary deployments are a popular tool to reduce risk when deploying software, by exposing a new version to a small subset of traffic before rolling it out more broadly. Creating the machinery to do this kind of controlled rollout, and monitoring for possible problems and […]

In September, we announced the general availability of Amazon Corretto 17. Amazon Corretto is a no-cost, multi-platform, production-ready distribution of the Open Java Development Kit (OpenJDK). The JDK community has declared that OpenJDK 17 will be a long-term supported (LTS) version, which means it will continue to be updated beyond the standard two quarterly updates […]

This post was written by Frank Munz, Staff Developer Advocate at Databricks. An introduction to Delta Sharing During the past decade, much thought went into system and application architectures using domain-driven design and microservices, but we are still on the verge of building distributed data meshes. Such data meshes are based on two fundamental principles: […]