re:Cap part three – open source at re:Invent 2019

Wrapping up our final summary, we kick off with a roundup of the open source updates in the area of compute and emerging technologies. We start with a great explanation of Fargate on Firecracker from Clare Ligouri during Werner Vogel’s keynote, and proceed to a broad selection of the container sessions and workshops that ran throughout the week, finishing with a summary of other interesting open source-related sessions and workshops.

This is the final summary of open source news from re:Invent. In this, part three of three, and we will be covering compute and emerging technologies such as robotics and blockchain as well as covering all other areas of open source such as Java. Part one covered all things data, analytics and machine learning and and part two covered mobile web development, containers and devops.

Compute

Containers

During Werner Vogels’ keynote, Clare Liguori provided a peek at Firecracker containerd, the future of how we will be running Fargate; check it out on GitHub and, while you’re there, why not have a look around and join the community? You can also watch the announcement during the keynote, Clare’s piece starts at around 21 min.

Sessions

Container fans, put these sessions on your watch list:

• CON212–Running Kubernetes at Amazon Scale using Amazon EKS – A great session to get you started, introducing many of the core concepts that you will see throughout the other sessions.
• CON329–Running Kubernetes containers at scale (with Square) – A deep dive into one of our customers (Square) to learn how they built on top of Amazon EKS and how this has allowed them to increase developer velocity and focus on security.
• CON334–Running high security workloads on Amazon EKS – If you wanted to know how to run security-sensitive workloads on containers, this is the session for you. It’s a deep dive exploring a real-life customer solution and how they enabled mission-critical, security-sensitive workloads on Amazon EKS.
• CON421–Amazon EKS under the hood – A very in-depth look at the architecture, operations, and design of Amazon EKS. This is about as deep as it gets.
• OPN211–How Zalando runs Kubernetes clusters at scale – Another customer story, showing how Zalando does continuous delivery, and how it open sourced tooling to improve the developer experience and create efficiency.

Workshops

• Amazon EKS workshop

Firecracker

Learn more about Firecracker and how AWS partners such as Weaveworks are able to innovate by contributing to this project. Some interesting stats about Firecracker:

• As of November 22nd, we have 100 contributions from 48 open source contributors to the project (18% of total contributions) as well as bug and feature requests and RFC feedback.
• AWS Lambda runs trillions of invocations per month on top of Firecracker.

In OPN402–Firecracker open source innovation you can learn more about Firecracker and how customers are building innovative new solutions on top of it.

Linux

Are you running or managing Linux desktops? Have you been exploring how to provide and provision these as virtual desktops? EUC330–How to use Amazon Linux Workspaces for agile development provides an overview of Amazon’s Linux Workspaces for Enterprise, and how to prepare and how to build custom images for your users. To learn more about the Amazon Linux roadmap, see the last part of CON217–Roadmaps for containers, application networking and Amazon Linux (starts at 40:00).

Serverless

SVS341–An in–depth tour of AWS SAM – Get to know the Serverless Application Model (SAM) in this great session, expertly delivered by Alex Wood. If that whets your appetite and you want an in–depth demo, check out DEM158–AWS SAM toolkit: build, test and debug your serverless applications.

Blockchain

Learn more about our approach to enterprise blockchain and how we leverage open source projects such as Hyperledger. In OPN217–Enterprise Blockchain: AWS’s open source approach the speakers address what is blockchain, do you need a blockchain, and Amazon’s view of blockchain. Tamara Dull is joined by Brian Behlendorf, Executive Director of the Hyperledger Foundation.

Robotics

In the ROB201–Leadership session: Welcome to the era of smart robots, Roger Barga from AWS and Chris Morgan from Toyota Advanced Robotics introduce robotics to the beginner, outline how AWS RoboMaker can bring modern application development techniques to help accelerate robotics development, and then Chris walks us through some great use cases and examples of robotics today. A few other AWS RoboMaker sessions and announcements you should check out:

• Learn more about ROS2, what the key changes are since ROS1, and some practical guidance on how to approach developing your robotics applications in OPN201–Change is coming to robotics development: The shift to ROS2.
• Learn more about trends and innovations in robotics technologies, covering drones, robotic arms, and unmanned ground vehicles in ROB302–Innovation in robotics; Insights from NASA JPL and AWS.
• We also launched a great open source challenge with JPL, the AWS JPL Open Source Rover challenge.

Workshops

• Voice Robotics workshop
• AWS RoboMaker workshop and the code for those workshops can be found at https://github.com/jerwallace/aws–robomaker–workshops

Other open source sessions and announcements of interest

Security and governance

We had some great security-based material:

• OPN215 – Intelligent Automation with AWS and Snort IDS – combining open source security tooling, AWS automation and machine learning services to better protect your resources. This was a builder session, but Sam has made the source code available.
• OPN305–How Optum manages transient developer accounts at scale – learn how Optum created this open source project to help power thousands of its builders with its Disposable Cloud Environments (DCE).
• If you have or are thinking about setting up an open source program office, are are interested in how to approach managing open source projects, whether consuming or contributing, we had a great session on the importance of understanding how to think about security. In this must-see session from Colm MacCárthaigh you will learn best practices to ensure that security is your number one priority when working with open source software. OPN219–it’s always day zero: Working on open source and security.

Internet of Things

Amazon FreeRTOS was covered in both IOT214–Whats new with AWS IoT Device software and IOT310–Bringing intelligence to the edge for connected home use cases.

Java

In OPN304–Migrating AWS Lambda’s front end from Java 8 to Java 11 learn about the journey of migrating AWS Lambda’s front–end service from Java 8 to Java 11 using Amazon Corretto, our no–cost distribution of OpenJDK. We also walk through the code and dependency changes required to migrate to Java 11, how we measured performance improvements, and how we safely deployed such a significant update to a large–scale service across multiple regions in production.

BPF

In OPN303 – BPF performance analysis at Netflix, Brendan Gregg spoke about Extended BPF (eBPF or simply BPF), an open-source Linux technology powering a whole new class of software: mini programs that run on events. BPF can be used to instrument arbitrary functions, as Brendan demoed with a BPF theremin.

BPF can also be used to create powerful performance analysis tools capable of analyzing CPUs, memory, disks, file systems, networking, languages, applications, and more. The session tours BPF tracing capabilities, including many new open-source performance analysis tools he developed for his new book BPF Performance Tools: Linux System and Application Observability. The talk also includes examples of using these tools in the Amazon Elastic Compute Cloud (Amazon EC2) cloud.

Keep up to date with open source at AWS

I hope this summary has been useful. I have looked for all the session videos that have been uploaded to date, but if I have missed anything, please get in touch and I will update this summary. Remember to check out the Open Source homepage to keep up to date with all our activity in open source by following us on @AWSOpen.