AWS Open Source Blog

Category: Open Source

An Open Distro for Elasticsearch cluster with balanced resource usage

Use Elasticsearch’s _rollover API For efficient storage distribution

Many Open Distro for Elasticsearch users manage data life cycle in their clusters by creating an index based on a standard time period, usually one index per day. This pattern has many advantages: ingest tools like Logstash support index rollover out of the box; defining a retention window is straightforward; and deleting old data is […]

Add Single Sign-On to Open Distro for Elasticsearch Kibana Using SAML and ADFS

Open Distro for Elasticsearch Security (Open Distro Security) comes with authentication and access control out of the box. Prior posts have discussed LDAP integration with Open Distro for Elasticsearch and JSON Web Token authentication with Open Distro for Elasticsearch. Security Assertion Markup Language 2.0 (SAML) is an open standard for exchanging identity and security information […]

Diagram showing where PartiQL fits with other data sources.

Announcing PartiQL: One query language for all your data

Data is being gathered and created at rates unprecedented in history. Much of this data is intended to drive business outcomes but, according to the Harvard Business Review, “…on average, less than half of an organization’s structured data is actively used in making decisions…” The root of the problem is that data is typically spread […]

AWS Parallel Cluster graphic

AWS ParallelCluster with AWS Directory Services Authentication

AWS ParallelCluster simplifies the creation and deployment of HPC clusters. In this post we combine ParallelCluster with AWS Directory Services to create a multi-user, POSIX-compliant system with centralized authentication and automated home directory creation. To grant only the minimum permissions to the nodes in the cluster, no AD configuration parameters or permissions are stored directly […]

Best Practices for Running Ansys Fluent Using AWS ParallelCluster

Using HPC (high performance computing) to solve Computational Fluid Dynamics (CFD) challenges has become common practice. As the growth from HPC workstation to supercomputer has slowed over the last decade or two, compute clusters have increasingly taken the place of single, big SMP (shared memory processing) supercomputers, and have become the ‘new normal’. Another, more […]

diagram: configure Gravitational’s ( Teleport ( as an authentication proxy for Amazon Elastic Kubernetes Service (Amazon EKS ( using GitHub as the identity provider for authenticating users.

Authenticating to EKS Using GitHub Credentials with Teleport

July 15, 2020 update: Gravitational has updated the instructions for using Teleport with EKS to account for the latest changes in both products. Please see the Gravitational documentation for further details.  This post describes how to configure Gravitational’s Teleport as an authentication proxy for Amazon Elastic Kubernetes Service (Amazon EKS), using GitHub as the identity […]

eksctl – the EKS CLI

When we launched Amazon EKS, we had a plan for a more complete command line. We were intrigued by Weaveworks’ simultaneous launch of the open source command line tool eksctl, and excited about the user feedback we were hearing. We decided, instead of building our own, to embrace eksctl as part of the EKS planning […]

Brian Gerkey and William Woodall of Open Robotics – cake-cutting ceremony.

ROS by the Bay Meetup

by Camilo Buscaron Last month, AWS hosted at our East Palo Alto offices the second ROS by the Bay Meetup organized by Open Robotics. ROS is an open source set of software libraries and tools that which helps developers build robot applications. We were excited to support this growing community of open source developers, with over […]

AWS service that terminates TLS at the host level: graph showing dramatically reduced CPU utilization after deploying ACCP.

Amazon Introduces Amazon Corretto Crypto Provider (ACCP)

In October, 2018, we introduced Amazon Corretto, an open source, no-cost, multi-platform, production-ready distribution of the Open Java Development Kit (OpenJDK). At launch, we were focused on creating a high-quality, long-term supported distribution, with a few performance improvements. Today, we are pleased to release a major performance improvement feature: the Amazon Corretto Crypto Provider (ACCP). […]

Setup of the centralized logging demo app

Centralized Container Logging with Fluent Bit

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. Visit the website to learn more. by Wesley Pettit and Michael Hausenblas AWS is built for builders. Builders are always looking for ways to optimize, and this applies to application logging. Not all logs are of equal importance. Some require real-time analytics, […]