AWS Compute Blog
Category: AWS Batch
Introducing Spot Blueprints, a template generator for frameworks like Kubernetes and Apache Spark
This post is authored by Deepthi Chelupati, Senior Product Manager for Amazon EC2 Spot Instances, and Chad Schmutzer, Principal Developer Advocate for Amazon EC2 Customers have been using EC2 Spot Instances to save money and scale workloads to new levels for over a decade. Launched in late 2009, Spot Instances are spare Amazon EC2 compute […]
Read MoreIntroducing retry strategies for AWS Batch
This post is contributed by Christian Kniep, Sr. Developer Advocate, HPC and AWS Batch. Scientists, researchers, and engineers are using AWS Batch to run workloads reliably at scale, and to offload the undifferentiated heavy lifting in their day-to-day work. But even with a slight chance of failure in the stack, the act of mitigating these […]
Read MoreHow to run 3D interactive applications with NICE DCV in AWS Batch
This post is contributed by Alberto Falzone, Consultant, HPC and Roberto Meda, Senior Consultant, HPC. High Performance Computing (HPC) workflows across industry verticals such as Design and Engineering, Oil and Gas, and Life Sciences often require GPU-based 3D/OpenGL rendering. Setting up drivers and applications for these types of workflows can require significant effort. Similar GPU […]
Read MoreCustom logging with AWS Batch
This post was written by Christian Kniep, Senior Developer Advocate for HPC and AWS Batch. For HPC workloads, visibility into the logs of jobs is important to debug a job which failed, but also to have insights into a running job and track its trajectory to influence the configuration of the next job or terminate […]
Read MoreUsing AWS ParallelCluster serverless API for AWS Batch
In this post, I show how to integrate the AWS Batch CLI by AWS ParallelCluster with API Gateway.
Read MoreOrchestrating an application process with AWS Batch using AWS CloudFormation
This post is written by Sivasubramanian Ramani In many real work applications, you can use custom Docker images with AWS Batch and AWS CloudFormation to execute complex jobs efficiently. This post provides a file processing implementation using Docker images and Amazon S3, AWS Lambda, Amazon DynamoDB, and AWS Batch. In this scenario, the user […]
Read MoreOptimizing for cost, availability and throughput by selecting your AWS Batch allocation strategy
This post is contributed by Steve Kendrex, Senior Technical Product Manager, AWS Batch Introduction AWS offers a broad range of instances that are advantageous for batch workloads. The scale and provisioning speed of AWS’ compute instances allow you to get up and running at peak capacity in minutes without paying for downtime. Today, I’m […]
Read MoreLeveraging Elastic Fabric Adapter to run HPC and ML Workloads on AWS Batch
Leveraging Elastic Fabric Adapter to run HPC and ML Workloads on AWS Batch This post is contributed by Sean Smith, Software Development Engineer II, AWS ParallelCluster & Arya Hezarkhani, Software Development Engineer II, AWS Batch and HPC On August 2, 2019, AWS Batch announced support for Elastic Fabric Adapter (EFA). This enables you to run highly […]
Read MoreCreating an AWS Batch environment for mixed CPU and GPU genomics workflows
This post is courtesy of Lee Pang – AWS Technical Business Development I recently worked with a customer who needed to process a bunch of raw sequence files (FastQs) into Hi-C format (*.hic), which is used for the structural analysis of DNA/chromatin loops and sequence accessibility. The tooling they were interested in using was the Juicer […]
Read MoreGPU workloads on AWS Batch
Contributed by Manuel Manzano Hoss, Cloud Support Engineer I remember playing around with graphics processing units (GPUs) workload examples in 2017 when the Deep Learning on AWS Batch post was published by my colleague Kiuk Chung. He provided an example of how to train a convolutional neural network (CNN), the LeNet architecture, to recognize handwritten digits […]
Read More