AWS Open Source Blog

Category: Technical How-to

How the Bottlerocket build system works

Bottlerocket is an open source, special-purpose operating system designed for hosting Linux containers, which was launched in 2020. As I delved into the Bottlerocket build system for a deeper understanding, I found it helpful to describe the system in detail (a form of rubber-duck debugging). This article is the result of my exploration and will […]

Read More
pickup – stock.adobe.com

Leverage deep learning in Scala with GPU on Spark 3.0

This post was contributed by Qing Lan, Carol McDonald, and Kong Zhao. With the growing interest in deep learning (DL), more users are using DL in their production environments. Because DL requires intensive computational power, developers are leveraging GPUs to do their training and inference jobs. As part of a major Apache Spark initiative to […]

Read More
olegkruglyak3 – stock.adobe.com

How Netflix uses Deep Java Library (DJL) for distributed deep learning inference in real-time

This post was written by Stanislav Kirdey, Lan Qing, Lai Wei, and Lu Huang. Netflix is one of the world’s largest entertainment services and it supports more than 190 million subscribers from different countries viewing from more than a thousand types of devices every day. One of the ways Netflix is able to sustain a […]

Read More
radachynskyi – stock.adobe.com - stacks of blocks

Improving zlib-cloudflare and comparing performance with other zlib forks

We worked with the maintainers of the Cloudflare fork of zlib (zlib-cloudflare) to improve the decompression performance on Arm and x86. With the changes, at level 6: On Arm: Compression performance: ~90 percent faster than zlib-madler (original zlib). Decompression performance: ~52 percent faster than zlib-madler. On x86: Compression performance: ~113 percent faster than zlib-madler. Decompression […]

Read More
tippapatt – stock.adobe.com

Testing AWS Lambda functions written in Java

Testing is an essential task when building software. Testing helps improve software quality by finding bugs before they reach production. The sooner we know there is a defect in code, the easier and cheaper it is to correct. Automated tests are a central piece in reducing this feedback loop. In association with a continuous integration […]

Read More

Remote visualization in HPC using NICE DCV with ParallelCluster

NICE DCV is an AWS-owned high performance remote display protocol, which specializes in 2D/3D interactive streaming over the internet or a local network (e.g., WiFi). With the power of NICE DCV we can seamlessly connect to our remote session running either in the cloud or data center via internet from a local laptop. We can […]

Read More

Deploy fast.ai-trained PyTorch model in TorchServe and host in Amazon SageMaker inference endpoint

Over the past few years, fast.ai has become one of the most cutting-edge, open source, deep learning frameworks and the go-to choice for many machine learning use cases based on PyTorch. It has not only democratized deep learning and made it approachable to general audiences, but fast.ai has also become a role model on how […]

Read More

Using Kedro pipelines to train Amazon SageMaker models

Machine learning (ML) and artificial intelligence (AI) adoption is growing at nearly 25 percent per year in a variety of businesses, which results in data scientists and engineers building more analytical models per person with similar levels of resources as last year. To keep up with such high demand, builders need to remove manual and […]

Read More
AWS Open Source logo.

Migrating Cortex CI/CD workflows to GitHub Actions

In this blog post, intern engineers Azfaar Qureshi and Shovnik Bhattacharya talk about their experience working with Cortex, a popular open source observability project. They share the challenges they faced and how they applied lessons learned to improve the development experience for other contributors in the Cortex Project. The rise of open source has completely […]

Read More

Launching the AWS Distro for OpenTelemetry developer site with Gatsby and GraphQL

In this post, AWS intern Wilbert Guo shares his experience in building the AWS Distro for OpenTelemetry developer site using Gatsby and GraphQL. The developer site aims to provide a place where customers can find out more information about the project, as well as get involved and download the distribution. OpenTelemetry is a popular open […]

Read More