Amazon EKS now supports Kubernetes 1.23

Introduction

The Amazon Elastic Kubernetes Service (Amazon EKS) team is happy to announce support for Kubernetes 1.23. Amazon EKS and Amazon EKS Distro can now run Kubernetes version 1.23, with support in Amazon EKS Anywhere launching soon after.

The upstream project theme for this release is “The Next Frontier.” Speaking with Kubernetes 1.23 Release Lead, Rey Lejano, there were a few reasons behind the naming. Rey is a big Star Trek fan (we get that). Star Trek V: Final Frontier starts and ends in Yosemite National Park, “a place where I grew up going to,” Rey said. Additionally, “The release itself brings new and graduated enhancements encapsulating what’s ‘next’ for Kubernetes. Since each Release Team brings new contributors and for those folks, the Release Team and the release is their first contribution in their open-source frontier.”

Solution overview

This release has several important changes including the Pod Security admission controller moving to beta, updates to the Container Storage Interface (CSI) Migration for working with AWS Elastic Block Store (Amazon EBS) volumes, and deprecation and removals of certain beta application programming interfaces (APIs) and the features detailed below. Notably, the CSI driver updates require some customers to take actions to ensure a smooth upgrade process. Kubernetes 1.23 also introduces Pod Security Standards (PSS) and Pod Security Admission (PSA), as well as the GA of Horizontal Pod Autoscaler (HPA) v2. Thank you for all the work the upstream Kubernetes 1.23 Release Team did to bring this release to the greater cloud-native ecosystem.

Important changes for workloads with Amazon EBS volumes

There are some notable changes to the CSI in Kubernetes 1.23 that may impact your upgrade process if you’re using Amazon EBS volumes. The CSI was developed in Kubernetes to replace the in-tree driver mechanism, which was specialized code that existed within the Kubernetes project source code to work with specific storage providers, such as Amazon EBS in AWS. With the addition of CSI support, we now have a simplified plug-in model that makes it easier for storage providers like AWS to release features and maintain support without depending on the Kubernetes release cycle.

To help users migrate from the in-tree plugin to CSI drivers, the CSIMigrationAWS is now enabled by default in Kubernetes 1.23, translating the in-tree APIs to their equivalent CSI APIs when the Amazon EBS CSI driver is installed. If you’re using Amazon EBS volumes, you must install the Amazon EBS CSI driver prior to upgrading to 1.23. If the Amazon EBS CSI plugin is not installed, then the volume operations including provision, attach, mount, and resize fail. Refer to the documentation to for additional information.

Amazon EKS is removing support for Dockershim in version 1.24

The upstream Kubernetes project deprecated Dockershim in version 1.20 and removed it in 1.24. The Dockershim is the component that allows Kubernetes to use Docker as a container runtime. While the Dockershim is deprecated, customers can continue using Docker as a fully functional runtime in Amazon EKS 1.23. Users should migrate to containerd before support is finally removed in the Amazon EKS 1.24 release. If you still need to use Dockershim after EKS 1.24, you’ll have to build an Amazon Machine Image (AMI) for that purpose because it’ll no longer be included in the optimized AMI that Amazon EKS vends. For information on how to install and manage clusters with containerd, a graduated Cloud Native Computing Foundation (CNCF) project used by AWS Fargate, Bottlerocket, and Windows workers, please see the Kubernetes runtime changes and EKS section of the Amazon EKS 1.21 release blog.

Kubernetes 1.23 highlights

If you’re interested in all of the notable features, you should read the Kubernetes Blog release post and full release notes. We’ve highlighted some of the key updates here. For a complete list of changes see the Kubernetes 1.23 change log or watch the live webinar on the CNCF YouTube for a 40 minute run-down.

Pod Security Admission

The Pod Security admission controller moves to beta in the upstream Kubernetes 1.23, and is now enabled by default. Pod Security admission is a replacement for Pod Security Policies (PSPs), which were deprecated in version 1.21 and will be removed completely in 1.25. If you’re still using PSPs, now is a good time to plan for your next move. Pod Security defines three policies to broadly cover a number of use-cases for securing pods: privileged, baseline and restricted, from least to most restrictive. Pod Security admission offers a simple solution that is built into Kubernetes, but is not as flexible or fine-grained as policy-as-code solutions, like Kyverno, Open Policy Agent (OPA)/Gatekeeper, and more. To learn more about the best practices to implement Pod Security, see the Pod Security section in the Amazon EKS Best Practices guide.

HPA v2 graduates to stable

The HorizontalPodAutscaler autoscaling/v2 stable API moved to Generally Available (GA) in version 1.23, and subsequently the HPA autoscaling/v2beta2 API has been deprecated. When a Kubernetes API feature reaches GA, it is a commitment that the API stays in place throughout the current major version, and that the configuration structure won’t see any breaking changes. Although you have plenty of time, you should consider updating your manifest files to use autoscaling/v2. A notable feature in HPA v2 is the ability to scale on custom metrics, allowing the HPA controller to query the Kubernetes API to scale based on custom metrics not built into Kubernetes. You can also specify multiple metrics for the HPA controller to evaluate and scale. By the way, if you’re looking to extend the capabilities of HPA with a powerful event-driven autoscaler, look into Kubernetes Event-driven Autoscaling (KEDA), a CNCF incubating project.

IPv6 mode vs dual stack

In Kubernetes 1.23, dual-stack IPv4/IPv6 networking is now GA, allowing for pods and services to be assigned addresses from both IPv4 and IPv6. However, Amazon EKS support for IPv6 (introduced in 1.21) focuses on resolving the IPv4 exhaustion problem from the limited IPv4 address space, which is a primary issue that Amazon EKS customers facing today. While dual-stack may help with migration use-cases, pods/services still consume an IPv4 address. With Amazon EKS in IPv6 mode, pods receive IPv6 addresses while maintaining the ability to route to IPv4 endpoints. To learn more about IPv6 mode and how it is distinct from the upstream dual-stack support, see our IPv6 section in the documentation.

Ephemeral containers is beta

Ephemeral containers are now beta in Kubernetes 1.23, and are enabled by default. With ephemeral containers, you can now troubleshoot pods by deploying a temporary debugging container. It can be challenging to debug pods when kubectl exec fails due to the lack of a shell (distroless images) or when a container has crashed. Now you can use the kubectl debug command to deploy a temporary container into the pod to run debugging commands. Notably, this command allows you to share the process namespace of another container running in the pod, whereas simply doing a kubectl run to run another container in the pod won’t allow you to access that process namespace. Learn more about using ephemeral containers in the Kubernetes docs site.

CPU Manager policies

CPU Manager Policy Options has graduated to beta in Kubernetes 1.23, meaning you can now configure additional options to fine tune the behavior of CPU Manager policies. With the default CPU allocation strategy, containers can be allocated as individual virtual cores, which can lead to containers sharing the same physical cores that potentially causing noisy neighbor issues. With 1.23, you can now configure the full-pcpus-only policy, which guarantees that pods will only be started when their central processing unit (CPU) requests can be fulfilled with full physical cores. Learn more in the CPU Management Policies section of the Kubernetes docs.

Windows Privileged Containers

Linux containers in Kubernetes have been able to run in privileged mode, which allows them to access the host operating system for administrative capabilities. This feature is now available in 1.23 for Windows containers by setting the windowsOptions.hostProcess flag in the Pod specification. Although this mode is not recommended for most workloads, it can be helpful for certain security or monitoring purposes. Note that HostProcess pods run directly on the host, and all containers in these pods must run as HostProcess containers. Learn more about privileged mode for containers on the Kubernetes docs site.

Conclusion

Upgrading EKS

You can learn about how to upgrade your Amazon EKS version on our documentation. While Amazon EKS and Amazon EKS Distro can now run version 1.23, Amazon EKS Anywhere support will follow soon after. If you’re using Amazon EBS volumes, then read the relevant section above and install the Amazon EBS CSI driver prior to upgrading to Amazon EKS 1.23.

If you’re using Kubernetes 1.19 or 1.20, please migrate to a newer version as end of support for version 1.19 was August 1st, 2022, and end of support for version 1.20 is November 1st, 2022. . You can refer to the complete Amazon EKS release calendar in our documentation.