AWS Open Source Blog

Category: Open Source

Open source tools are scaling and expanding access to education

In recent months, open source technologies have helped universities and colleges around the world scale remote learning at an unprecedented rate. As educators sought online resources and collaboration tools, open source technologies provided a unique opportunity to scale their education in a fast and cost-effective way without compromising flexibility or quality. Additionally, these tools provided […]

Dgraph on AWS: Setting up a horizontally scalable graph database

This article is a guest post from Joaquin Menchaca, an SRE at Dgraph. Dgraph is an open source, distributed graph database, built for production environments, and written entirely in Go. Dgraph is fast, transactional, sharded, and distributed (joins, filters, sorts), consistently replicated with Raft, and provides fault tolerance with synchronous replication and horizontal scalability. The […]

AWS adds observability metrics to the OpenTelemetry C++ library

In this post, three AWS interns—Brandon Kimberly, Ankit Bhargava, and Hudson Humphries—describe their first engineering contributions to the popular open source observability project OpenTelemetry. Recently we made contributions to OpenTelemetry that included the metrics collection and processing functionality for the C++ library. These metrics are collected from instrumented applications and infrastructure. They allow users to […]

Announcing the General Availability of Bottlerocket, an open source Linux distribution built to run containers

As our customers increasingly adopt containers to run their workloads, we saw a need for a Linux distribution designed from the ground up to run containers with a focus on security, operations, and manageability at scale. Customers needed an operating system that would give them the ability to manage thousands of hosts running containers with […]

How TalkingData leverage DJL with PyTorch for Large-Scale Offline Inference

How TalkingData uses AWS open source Deep Java Library with Apache Spark for machine learning inference at scale

This post is contributed by Xiaoyan Zhang, a Data Scientist from TalkingData. TalkingData is a data intelligence service provider that offers data products and services to provide businesses insights on consumer behavior, preferences, and trends. One of TalkingData’s core services is leveraging machine learning and deep learning models to predict consumer behaviors (e.g., likelihood of […]

Managing AWS ParallelCluster SSH users with OpenLDAP

A common request from AWS ParallelCluster users is to have the ability to deploy multiple POSIX user accounts. The wiki on the project GitHub page documents a simple mechanism for achieving this, and a previous blog post, “AWS ParallelCluster with AWS Directory Services Authentication,” documents how to integrate AWS ParallelCluster with AWS Directory Service. However, […]

Building resilient services at Prime Video with chaos engineering

Large-scale distributed software systems are composed of several individual sub-systems—such as CDNs, load balancers, and databases—and their interactions. These interactions sometimes have unpredictable outcomes caused by unforeseen turbulent events (for example, a network failure). These events can lead to system-wide failures. Chaos engineering is the discipline of experimenting on a distributed system to build confidence […]

Improving the serverless developer experience with sls-dev-tools

This post was contributed by AWS Serverless Hero Ben Ellerby. One of the most exciting things about serverless is the acceleration it can provide to teams. No longer do you have to invest time in the undifferentiated, heavy lifting of managing the infrastructure of our applications. Instead you can spend time writing code that creates […]

Cost tracking for OpenShift on AWS

AWS provides a collection of tools and services to give customers the ability to manage the resources within their AWS accounts. In this article I will briefly explore some of these tools and services, as well as an open source project that can be used to integrate AWS cost-management services and features directly into Red […]