AWS Open Source Blog

Category: Technical How-to

Creating simple AWS Cost and Usage charts with D3 JavaScript library

Web applications interacting with AWS in a number of ways may need to represent and display sets of information in the form of charts, diagrams, or graphs. Common examples of that information includes small amounts of data coming from AWS Costs & Usage Reports or Amazon Elastic Compute Cloud (Amazon EC2), either historical or real-time. […]

Read More

How to build a scalable BigBlueButton video conference solution on AWS

BigBlueButton is an open source video conference system that supports various audio and video formats and allows the use of integrated video-, screen- and document-sharing functions. BigBlueButton has features for multi-user whiteboards, breakout rooms, public and private chats, polling, moderation, emojis, and raise-hands. In this post, we will explain how AWS customers who are looking […]

Read More

OpenShift 4 on AWS Quick Starts

Customers selecting AWS as their preferred cloud have been able to deploy OpenShift through a variety of means. In this post we will explore what has changed in the latest version of OpenShift, and what these changes mean for AWS customers. We also will explore the latest OpenShift on AWS Quick Starts open source project, […]

Read More

Dgraph on AWS: Setting up a horizontally scalable graph database

This article is a guest post from Joaquin Menchaca, an SRE at Dgraph. Dgraph is an open source, distributed graph database, built for production environments, and written entirely in Go. Dgraph is fast, transactional, sharded, and distributed (joins, filters, sorts), consistently replicated with Raft, and provides fault tolerance with synchronous replication and horizontal scalability. The […]

Read More

AWS adds observability metrics to the OpenTelemetry C++ library

In this post, three AWS interns—Brandon Kimberly, Ankit Bhargava, and Hudson Humphries—describe their first engineering contributions to the popular open source observability project OpenTelemetry. Recently we made contributions to OpenTelemetry that included the metrics collection and processing functionality for the C++ library. These metrics are collected from instrumented applications and infrastructure. They allow users to […]

Read More
How TalkingData leverage DJL with PyTorch for Large-Scale Offline Inference

How TalkingData uses AWS open source Deep Java Library with Apache Spark for machine learning inference at scale

This post is contributed by Xiaoyan Zhang, a Data Scientist from TalkingData. TalkingData is a data intelligence service provider that offers data products and services to provide businesses insights on consumer behavior, preferences, and trends. One of TalkingData’s core services is leveraging machine learning and deep learning models to predict consumer behaviors (e.g., likelihood of […]

Read More

Managing AWS ParallelCluster SSH users with OpenLDAP

A common request from AWS ParallelCluster users is to have the ability to deploy multiple POSIX user accounts. The wiki on the project GitHub page documents a simple mechanism for achieving this, and a previous blog post, “AWS ParallelCluster with AWS Directory Services Authentication,” documents how to integrate AWS ParallelCluster with AWS Directory Service. However, […]

Read More

Building resilient services at Prime Video with chaos engineering

Large-scale distributed software systems are composed of several individual sub-systems—such as CDNs, load balancers, and databases—and their interactions. These interactions sometimes have unpredictable outcomes caused by unforeseen turbulent events (for example, a network failure). These events can lead to system-wide failures. Chaos engineering is the discipline of experimenting on a distributed system to build confidence […]

Read More
workflow: how to deploy TorchServe on an Amazon EKS cluster for inference, which will allow you to quickly deploy a pre-trained machine learning model as a scalable, fault-tolerant web-service for low latency inference

Running TorchServe on Amazon Elastic Kubernetes Service

This article was contributed by Josiah Davis, Charles Frenzel, and Chen Wu. TorchServe is a model serving library that makes it easy to deploy and manage PyTorch models at scale in production environments. TorchServe removes the heavy lifting of deploying and serving PyTorch models with Kubernetes. TorchServe is built and maintained by AWS in collaboration […]

Read More
Kubeflow logo surrounded by AWS logos

Enterprise-ready Kubeflow: Securing and scaling AI and machine learning pipelines with AWS

Many AWS customers are building AI and machine learning pipelines on top of Amazon Elastic Kubernetes Service (Amazon EKS) using Kubeflow across many use cases, including computer vision, natural language understanding, speech translation, and financial modeling. In this post, we will describe AWS contributions to the Kubeflow project, which provide enterprise readiness for Kubeflow deployments. […]

Read More