AWS Open Source Blog
AWS Open Source blog in review: 2019
Last year I published a list of the open source blog’s 2018’s top ten posts. This year I’m starting with a list of lists of things that happened on the AWS Open Source blog in 2019.
Launches
We launched new projects, features, and project support:
- Open Distro for Elasticsearch (more info below)
- Amazon Introduces Amazon Corretto Crypto Provider (ACCP)
- Announcing PartiQL: One query language for all your data
- Announcing Gluon Time Series, an Open-Source Time Series Modeling Toolkit
- Centralized Container Logging with Fluent Bit
- Building Spinnaker Features for Amazon ECS
- Continuous delivery of container applications to AWS Fargate with GitHub Actions
- The Sumo Logic integration with Amazon EKS
- Code and beyond: How we contribute to the Apache Cassandra community
Graduated
Some of our new projects, services, and commitments, after a launch and incubation period here on the Open Source blog, have “graduated” to blogs of their own:
Robotics
This blog’s involvement with robotics started at re:Invent 2018, when we announced AWS Robomaker and our participation in the Robot Operating System (ROS) community. This was followed by more posts about ROS and robotics, particularly after Cam Buscaron joined the AWS Open Source marketing team as an evangelist for robotics. AWS Robotics now has its own blog.
Open Distro for Elasticsearch
This open source project was launched last March with a post from Adrian Cockcroft on Keeping Open Source Open. Between March and September, Jon Handler managed and edited many more posts about Open Distro into publication on the Open Source blog, as well as writing many himself. Topics ranged from Get Up and Running with Open Distro for Elasticsearch to release announcements to Running Open Distro for Elasticsearch on Kubernetes (our longest post ever). You can review all posts on Open Distro for Elasticsearch on the AWS Open Source blog. In September, Open Distro for Elasticsearch got its own blog, and in future you’ll find most of the material on Open Distro over there.
Containers
Amazon Elastic Kubernetes Service was announced in preview during re:Invent 2017, and became generally available in June 2018. Over 2018 and 2019, the Open Source blog published many posts about various aspects of running containers on AWS.
Features:
- Fine-grained IAM roles for service accounts
- Using Pod Security Policies with Amazon EKS Clusters
- Horizontal Pod Autoscaling for Amazon EKS
- Improvements for Amazon EKS Worker Node Provisioning
In-depth how-to’s:
- Using a Network Load Balancer with the NGINX Ingress Controller on Amazon EKS
- Best Practices for Optimizing Distributed Deep Learning Performance on Amazon EKS
We also covered using many open source projects on Amazon EKS, in posts often written by the creators of these projects: Istio, OpenFaaS, Kubeflow, Kube-OIDC-Proxy, Jenkins and HashiCorp Terraform, Spinnaker, Fairwinds Polaris, Teleport, eksctl, Prometheus, FSx for Lustre CSI Driver, Kiali, TriggerMesh KLR, Rancher, Open Policy Agent, mu, Kubernetes Service Catalog, Kubernetes Ingress, AWS Service Operator, CNI Metrics Helper, GitKube, Helm, AWS-IAM-Authenticator, Kubeless, kube-aws, EBS CSI driver.
The AWS Containers blog was launched in October, 2019 and you’ll now find a lot of the EKS and containers-related material there, but you’ll also continue to see posts at the intersect of open source, containers, and EKS here on the Open Source blog.
Other Amazon open source projects
Open source projects still being covered more or less exclusively here include:
Other projects covered in 2019 included Building Your Own AWS Service Broker Services. But open source is a broad topic that can also be found on other AWS blogs, and you’ll find some listed under “Latest Posts About Open Source” on opensource.amazon.com.
Machine Learning
Most blog posts about machine learning live on the Machine Learning blog, but this year we hosted a few about machine learning and open source:
- Best Practices for Optimizing Distributed Deep Learning Performance on Amazon EKS
- Announcing Gluon Time Series, an Open-Source Time Series Modeling Toolkit
Customer projects
As in 2018, we featured a number of guest posts from customers about their projects that run on AWS and are potentially useful to other customers, as well as posts from AWS colleagues about how to use open source on AWS. In addition to the large list above of projects that work with EKS, we published:
- Integrating Phabricator with AWS CodePipeline via AWS CodeCommit
- Kata Containers 1.5 Release with Support for Firecracker
- Observe Your Service Mesh with Kiali
- Deploying the AWS IAM Authenticator to kops
- Build a Deployment Pipeline with Spinnaker on Kubernetes
- AWS Cloud Map service discovery for serverless applications
Another kind of “graduation”
Last year Ilya Dmitrichenko of Weaveworks and Chris Hein explained how eksctl “gives you a simple, single, one-line command to bring up a cluster with a basic VPC, and completes the process by writing a new KUBECONFIG
and deploying the aws-auth
ConfigMap
, allowing you to get up and running with EKS in minutes.“ This year ”We decided… to embrace eksctl
as part of the EKS planning cycle, and to encourage others to contribute as well – and we’ve been happy we did!“
Foundations, events, and support for open source projects
In October, we announced AWS Promotional Credits for Open Source Projects.
We also aimed to keep readers informed about our participation in open source foundations and events:
- Supporting The Apache Software Foundation
- AWS Joins the GraphQL Foundation
- AWS Joins the .NET Foundation
- Amazon joins the Java Community Process (JCP)
- AWS’ Sponsorship of the Rust Project
We participated in and sponsored many open source events. You can find a full list on opensource.amazon.com. We published blog posts about just a few:
- Open Source Love at FOSDEM 2019
- Amazonians at OSCON 2019: A Week of Open Source
- ROS by the Bay Meetup
- Open source at AWS re:Invent 2019 – re:Invent featured an open source track for the first time, in addition to many open source-related sessions in other tracks.
Open source know-how
We shared some of our own knowledge about open source:
Late in the year, we were thrilled to have Matt Asay join AWS. In addition to writing the new Modernizing with AWS blog, he has been writing on open source topics for the Open Source blog:
- Why does AWS contribute to open source? The Firecracker example
- How AWS is helping to open source the future of robotics
- An outsider’s inside view on open source at AWS
- Code and beyond: How we contribute to the Apache Cassandra community
Yes, we’ll also do a top ten posts list for 2019 – watch this space!