AWS Compute Blog

Category: AWS Batch

application using AWS Batch

Orchestrating an application process with AWS Batch using AWS CloudFormation

This post is written by Sivasubramanian Ramani In many real work applications, you can use custom Docker images with AWS Batch and AWS CloudFormation to execute complex jobs efficiently.   This post provides a file processing implementation using Docker images and Amazon S3, AWS Lambda, Amazon DynamoDB, and AWS Batch. In this scenario, the user […]

Read More

Optimizing for cost, availability and throughput by selecting your AWS Batch allocation strategy

This post is contributed by Steve Kendrex, Senior Technical Product Manager, AWS Batch   Introduction   AWS offers a broad range of instances that are advantageous for batch workloads. The scale and provisioning speed of AWS’ compute instances allow you to get up and running at peak capacity in minutes without paying for downtime. Today, I’m […]

Read More

Leveraging Elastic Fabric Adapter to run HPC and ML Workloads on AWS Batch

Leveraging Elastic Fabric Adapter to run HPC and ML Workloads on AWS Batch  This post is contributed by  Sean Smith, Software Development Engineer II, AWS ParallelCluster & Arya Hezarkhani, Software Development Engineer II, AWS Batch and HPC   On August 2, 2019, AWS Batch announced support for Elastic Fabric Adapter (EFA). This enables you to run highly […]

Read More

Creating an AWS Batch environment for mixed CPU and GPU genomics workflows

This post is courtesy of Lee Pang – AWS Technical Business Development  I recently worked with a customer who needed to process a bunch of raw sequence files (FastQs) into Hi-C format (*.hic), which is used for the structural analysis of DNA/chromatin loops and sequence accessibility. The tooling they were interested in using was the Juicer […]

Read More

GPU workloads on AWS Batch

Contributed by Manuel Manzano Hoss, Cloud Support Engineer I remember playing around with graphics processing units (GPUs) workload examples in 2017 when the Deep Learning on AWS Batch post was published by my colleague Kiuk Chung. He provided an example of how to train a convolutional neural network (CNN), the LeNet architecture, to recognize handwritten digits […]

Read More
Step Functions

Building Simpler Genomics Workflows on AWS Step Functions

This post is courtesy of Ryan Ulaszek, AWS Genomics Partner Solutions Architect and Aaron Friedman, AWS Healthcare and Life Sciences Partner Solutions Architect In 2017, we published a four part blog series on how to build a genomics workflow on AWS. In part 1, we introduced a general architecture highlighting three common layers: job, batch and […]

Read More

Building a tightly coupled molecular dynamics workflow with multi-node parallel jobs in AWS Batch

Contributed by Amr Ragab, HPC Application Consultant, AWS Professional Services and Aswin Damodar, Senior Software Development Engineer, AWS Batch At Supercomputing 2018 in Dallas, TX, AWS announced AWS Batch support for running tightly coupled workloads in a multi-node parallel jobs environment. This AWS Batch feature enables applications that require strong scaling for efficient computational workloads. Some of […]

Read More

Using Cromwell with AWS Batch

Contributed by W. Lee Pang and Emil Lerch, WWPS Professional Services DNA is often referred to as the “source code of life.” All living cells contain long chains of deoxyribonucleic acid that encode instructions on how they are constructed and behave in their surroundings. Genomics is the study of the structure and function of DNA […]

Read More

Deploy an 8K HEVC pipeline using Amazon EC2 P3 instances with AWS Batch

Update – April 14, 2020: AWS Elemental MediaConvert now supports 8K UHD video encoding. 8K encoding is available in the MediaConvert on-demand, professional tier, for resolutions up to 8192 x 4320 using HEVC encoding at 10-bit including HDR. To learn more, please visit https://aws.amazon.com/about-aws/whats-new/2019/11/8k-resolution-encoding-now-available-with-aws-elemental-media-convert/. Contributed by Amr Ragab, HPC Application Consultant, AWS Professional Services AWS provides several […]

Read More