AWS HPC Blog
Adding configurable namespaces, persistent volume claims, and other features for AWS Batch on Amazon EKS
With Re:Invent, holidays, and the general mayhem that the year end brings, you might have missed some announcements for AWS Batch on Amazon Elastic Kubernetes Service (Amazon EKS). The AWS Batch team recently released a number of features: configurable namespaces, persistent volume claims, support for sub-path in container volume mounts, and Pod annotations. Since releasing support for running jobs on your Amazon EKS clusters, the AWS Batch team works with customers, such as Balyasny Asset Management (BAM), on features that expands AWS Batch to broader sets of workloads. In particular, we heard from you that workload isolation, support for annotations, and more robust data access controls were the top asks. Let’s take a look at each of these new features and understand how they help manage complex containerized workloads with greater precision and security.
Configurable Kubernetes namespaces
Kubernetes users often use a namespace to define the permissions that a Pod can acquire both within the cluster and for access to AWS services through IAM roles for service accounts. Previously, AWS Batch compute environments defined a single namespace, and all jobs within the compute environment inherited the namespace’s associated permissions. With this new feature, you can configure a job’s Kubernetes namespace in the Pod’s metadata — either in the job definition or as an override when submitting a job using the eksPropertiesOverride
request property. If you do not specify a namespace, the pod uses the namespace set for the compute environment.
Persistent Volume Claims and mount sub-path support
Persistent Volumes and Persistent volume claims (PVCs) allow you to give access to persistent storage for a job. You define a PVC as part of an Amazon EKS job definition volume when you register or update the job definition. You can also define a sub-path for a container volume mount leveraging the new subPath
parameter within Amazon EKS container volume mounts.
Prior to PVC support, you needed to use host mounts on the node. Since host mounts are defined at node launch, you needed to coordinate what shared storage and what specific data was made available to all jobs that may run on the node, and what data and storage would specifically be available to a job at runtime. With PVCs you can now do all of that within the job definition, eliminating a risk where a Pod may gain access to data that it potentially should not have had access to. Overall, persistent volumes claims provide a more secure and deterministic way to access your data resources.
Shared configuration
As a reminder, AWS Batch uses your existing EKS clusters to launch jobs. There is shared configuration for AWS Batch and Kubernetes resources to work together. For example, to take advantage of additional namespaces when submitting jobs, you need to create the namespace and allow the AWS Batch service account to place resources within that namespace. For PVCs, you need to pre-define the persistent volume that will be claimed and configure the appropriate permissions for a job to claim it. If not correctly configured, a Pod might stay in a pending state indefinitely as the Kubernetes scheduler will keep trying to create the Pod’s volume claim. We recommend monitoring the EKS control plane for claim errors.
Both of these operational examples are true for other EKS cluster operations, and we recommend paying extra attention to resources and permissions when working with external cluster services, such as AWS Batch.
Pod annotations
Finally, we added support for Pod annotations. Annotations are key-value pairs used to attach arbitrary, non-identifying metadata to Kubernetes objects. You can use annotations to build integrations with external systems for Kubernetes resources. The annotations can be useful in various management and operational aspects, such as automation, monitoring, logging, and security. You can define annotations either when registering a job definition, or when you submit a job.
These external systems can leverage the annotations to get extra information about the Pod and act on that information. For example, a monitoring system can show that information in a dashboard, or a CI/CD system can use it to run a specific set of build pipelines. You can also use annotations within the running containers. For example, providing information on how to connect to a resource running outside of the Pod.
Generally, the difference between a Pod label and a Pod annotation is that annotations are not identifying. You can’t use an annotation to search for Kubernetes resources using the Kubernetes APIs. Specific to Batch, annotations can hold larger values and structured data. In Batch, labels are limited to 63 characters or less, and annotations can hold values up to 255 characters. Refer to the AWS Batch EKS metadata documentation for more information on syntax and value constraints for both annotations and labels.
Conclusion
These new features from AWS Batch for Amazon EKS improve workload management and data access control. By introducing configurable namespaces, persistent volume claims, container volume mount sub-paths, and Pod annotations, AWS Batch addressed key customer requirements around workload isolation and data security. Although these enhancements provide more granular control and flexibility, they also underscore the importance of proper and shared configuration between Amazon EKS and AWS Batch resources. These new features will help you more effectively manage complex containerized workloads with greater precision and security.
To try out the new features for Amazon EKS, check out the Getting started with AWS Batch on Amazon EKS user guide. If you have feedback, feel free to write us a note at ask-hpc@amazon.com.