Amazon EKS now supports Kubernetes version 1.24

The Amazon Elastic Kubernetes Service (Amazon EKS) team is pleased to announce support for Kubernetes version 1.24 for Amazon EKS and Amazon EKS Distro. We are excited for our customers to experience the power of the “Stargazer” release. Each Kubernetes release is given a name by the release team. The team chose “Stargazer” for this release to honor the work done by hundreds of contributors across the globe: “Every single contributor is a star in our sky, and Amazon EKS extends its sincere thanks to the upstream community and the Kubernetes 1.24 Release Team for bringing this release to the greater cloud-native ecosystem.”

Kubernetes 1.24 highlights

This post highlights the notable changes in Kubernetes 1.24 version. Refer to this Kubernetes blog post for the complete release notes. See the Kubernetes 1.24 change log for a full list of changes and deprecations.

Removal of Dockershim

The most significant change in this release is the removal of the Container Runtime Interface (CRI) for Docker (also known as Dockershim). Starting with version 1.24 of Kubernetes, the Amazon Machine Images (AMIs) provided by Amazon EKS will only support the containerd runtime. The EKS optimized AMIs for version 1.24 no longer support passing the flags enable-docker-bridge, docker-config-json, and container-runtime.

Important: Before upgrading your worker nodes to Kubernetes 1.24, you must remove all references to these flags.

The open container initiative (OCI) images generated by docker build tools will continue to run in your Amazon EKS clusters as before. As an end-user of Kubernetes, you will not experience significant changes. You can continue to use Docker to build your containers outside the cluster. Visit this link to learn why Amazon EKS is discontinuing Dockershim support. For more information, see Kubernetes is Moving on From Dockershim: Commitments and Next Steps on the Kubernetes Blog.

Detector for Docker Socket (DDS)

We recognize how challenging it can be to identify Dockershim usage in your environment. To help Amazon customers and the larger Kubernetes community, we have developed a tool: Detector for Docker Socket (DDS). You can use this tool to find out if your application is mounting docker socket and change it before you update your EKS clusters.

This detector tool is a kubectl plugin that can detect if any of your workloads or manifest files are mounting the docker.sock volume. If pods are part of a workload (e.g., Deployment, StatefulSet, etc.) it inspects the workload type instead of pods directly. The tool can also be used to look at Kubernetes manifest files on disk as well. Please refer to the project’s README for more details.

Beta APIs off by default

Beta APIs were enabled by default prior to this release. While this accelerated adoption of new features, it also introduced operational challenges for Kubernetes users, such as exposure to bugs and migration pain. The new beta APIs will not be enabled by default in clusters beginning with Kubernetes version 1.24. Existing beta APIs and new versions of existing beta APIs will continue to be enabled by default. Amazon EKS will follow the same behavior as upstream Kubernetes 1.24. You may refer to this link for more information regarding the rationale behind this change.

Topology Aware Hints

It is common practice to deploy Kubernetes workloads to nodes running across different availability zones (AZ) for resiliency and fault isolation. While this architecture provides great benefits, in many scenarios it will also result in cross-AZ data transfer charges. You may refer to this post to learn more about common scenarios for data transfer charges on EKS. Amazon EKS customers can now use Topology Aware Hints, which are enabled by default, to keep Kubernetes service traffic within the same availability zone. Topology Aware Hints provide a flexible mechanism to provide hints to components, such as kube-proxy, and use them to influence how the traffic is routed within the cluster.

Pod Security Policy

Pod Security Policy (PSP) was deprecated in Kubernetes version 1.21 and will be removed in version 1.25. PSPs are being replaced by Pod Security Admission (PSA), a built-in admission controller that implements the security controls outlined in the Pod Security Standards (PSS). PSA and PSS have both reached beta feature status as of Kubernetes version 1.23 and are enabled by default in Amazon Elastic Kubernetes Service (EKS). For guidance on implementing PSP and PSS, please review this blog post.

You can also leverage Policy-as-Code (PaC) solutions such as Kyverno, and OPA/Gatekeeper from the Kubernetes ecosystem as an alternative to PSA. Please visit the Amazon EKS Best Practices Guide for more information on PaC solutions and help deciding between PSA and PaC.

Simplified scaling for EKS Managed Node Groups (MNG)

For Kubernetes 1.24, we have contributed a feature to the upstream Cluster Autoscaler project that simplifies scaling the Amazon EKS managed node group (MNG) to and from zero nodes. Before, you had to tag the underlying EC2 Autoscaling Group (ASG) for the Cluster Autoscaler to recognize the resources, labels, and taints of an MNG that was scaled to zero nodes.

Starting with Kubernetes 1.24, when there are no running nodes in the MNG, the Cluster Autoscaler will call the EKS DescribeNodegroup API to get the information it needs about MNG resources, labels, and taints. When the value of a Cluster Autoscaler tag on the ASG powering an EKS MNG conflicts with the value of the MNG itself, the Cluster Autoscaler will prefer the ASG tag so that customers can override values as necessary.

Change to certificates controller

In Kubernetes 1.23 and earlier, kubelet serving certificates with unverifiable IP and DNS Subject Alternative Names (SANs) were automatically issued with the unverifiable SANs. Beginning with version 1.24, no kubelet-serving certificates will be issued if any SANs cannot be confirmed. This will prevent the kubectl exec and kubectl logs commands from working. Please follow the steps outlined in the EKS user guide to determine if you are impacted by this issue, the recommended workaround, and long-term resolution.

Kubelet Credential Provider Graduates to Beta

If you are using Amazon EKS Anywhere, Kubernetes 1.24 includes kubelet support for image credential providers. You can configure --image-credential-provider-config and --image-credential-provider-bin-dir flags for the kubelet to request credentials for a container registry dynamically, as opposed to storing static credentials on disk. Please refer to this link for a sample kubelet credential provider config file to use Amazon ECR plugins. Additionally, you can refer to the example implementation mentioned in this article to use Cognito User Pools as a managed identity provider and the Cognito binary as the credential provider.

Conclusion

We are excited for customers to start taking advantage of the numerous enhancements and new stable APIs in Kubernetes 1.24. Before updating to Kubernetes 1.24, you must modify the workloads to remove Dockershim dependencies and test using containerd. Also, keep in mind that Pod Security Policies will be removed in Kubernetes version 1.25. Please plan for migrating Pod Security Policies to  Pod Security Admission (PSA) or Policy-as-Code (PAC) solutions.

Amazon EKS provides support for at least four Kubernetes versions at any given time. Given the nature of Kubernetes release cycle, it is critical for all customers to have an ongoing upgrade plan. Amazon EKS support for version 1.20 reached end-of-life on November 1, 2022. Consequently, it is no longer possible to create new 1.20 clusters. You can see the Amazon EKS Kubernetes release calendar for more information. Visit EKS user guide to learn more about updating your EKS cluster to latest Kubernetes version.