AWS HPC Blog
Author: Angel Pizarro
Angel is a Principal Developer Advocate for HPC and scientific computing. His background is in bioinformatics application development and building system architectures for scalable computing in genomics and other high throughput life science domains.
Introducing “default” instance categories for AWS Batch
Today, we are launching a new set of instance family categories for AWS Batch, “default_x86_64” and “default_arm64″. These new categories represent both a clarification and an improvement upon the existing “optimal” instance type category. This blog post gives some background on the new feature and how you can configure your Batch environments to take advantage […]
Adding functionality to your applications using multiple containers in AWS Batch
Discover how to coordinate multiple applications in separate containers within a single AWS Batch job definition. Learn the benefits of this approach and how to share resources between containers for more efficient, scalable deployments.
How to use rate-limited resources in AWS Batch jobs with resource aware scheduling
Struggling with bottlenecks in your batch processing? AWS Batch’s new resource aware scheduling capability could be the solution your business needs. This feature allows you to define and manage consumable resources, helping maximize the use of your compute power. Check out our blog to learn more.
Using the Terraform AWS Cloud Control provider for managing AWS Batch resources
The Terraform AWS Cloud Control (AWSCC) provider now supports AWS Batch job definitions, enabling you to leverage recent and future enhancements to AWS Batch. Learn more in our latest blog post.
Adding configurable namespaces, persistent volume claims, and other features for AWS Batch on Amazon EKS
Exciting updates to AWS Batch on Amazon EKS! Configurable namespaces, persistent volume claims, and more. Check out our blog post to see how these features can help manage your complex containerized workloads.
Use Terraform to deploy a complete AWS Batch environment on Amazon EKS
Harness the power of AWS Batch on Amazon EKS with this new Terraform blueprint. It provides a complete template to create robust batch processing in the cloud. An easy button you shouldn’t miss.
Gang scheduling pods on Amazon EKS using AWS Batch multi-node processing jobs
AWS Batch multi-node parallel jobs can now run on Amazon EKS to provide gang scheduling of pods across nodes for large scale distributed computing like ML model training. More details here.
Job queue snapshots: see what’s at the head of your queues in AWS Batch
AWS Batch just grew a neat new feature: Job queue snapshots. This gives you the visibility you need for managing throughput in a dynamic environment – with competing priorities – and across multiple queues and workloads. Today we give you the inside scoop on how this feature works – especially when you’re using fair share scheduling.
Why you should use Fargate with AWS Batch for your serverless batch architectures
AWS Batch recently added support for Graviton and Windows containers on Fargate. Read about how these and other features like large task sizes and configurable local storage make AWS Batch on Fargate a fantastic serverless solution for your batch workloads.
Diving Deeper into Fair-Share Scheduling in AWS Batch
Today we dive into details of AWS Batch fair share policies and show how they affect job placement. You’ll see the result of different share policies, and hear about practical use cases where you can benefit from fair share job queues in Batch.