AWS HPC Blog

Tag: AWS Batch

Scalable and Cost-Effective Batch Processing for ML workloads with AWS Batch and Amazon FSx

Batch processing is a common need across varied machine learning use cases such as video production, financial modeling, drug discovery, or genomic research. The elasticity of the cloud provides efficient ways to scale and simplify batch processing workloads while cutting costs. In this post, you’ll learn a scalable and cost-effective approach to configure AWS Batch Array jobs to process datasets that are stored on Amazon S3 and presented to compute instances with Amazon FSx for Lustre.

Read More
AWS batch loves Amazon EFS - header image

Introducing support for per-job Amazon EFS volumes in AWS Batch

Large-scale data analysis usually involves some multi-step process where the output of one job acts as the input of subsequent jobs. Customers using AWS Batch for data analysis want a simple and performant storage solution to share with and between jobs. We are excited to announce that customers can now use Amazon Elastic File System (Amazon […]

Read More

Welcome to the AWS HPC Blog

This post is written by Deepak Singh, Vice President of Compute Services. At AWS, we love working with customers to solve their toughest challenges. High performance computing (HPC) is one of those challenges that pushes against the boundaries of AWS performance at scale. HPC is also a personal interest of mine, as I came to […]

Read More