AWS HPC Blog

Category: AWS Batch

Streamlining distributed ML workflow orchestration using Covalent with AWS Batch

Streamlining distributed ML workflow orchestration using Covalent with AWS Batch

Complicated multi-step workflows can be challenging to deploy, especially when using a variety of high-compute resources. Covalent is an open-source orchestration tool that streamlines the deployment of distributed workloads on AWS resources. In this post, we outline key concepts in Covalent and develop a machine learning workflow for AWS Batch in just a handful of steps.

Benchmarking the Oxford Nanopore Technologies basecallers on AWS

Oxford Nanopore sequencers enables direct, real-time analysis of long DNA or RNA fragments. They work by monitoring changes to an electrical current as nucleic acids are passed through a protein nanopore. The resulting signal is decoded to provide the specific DNA or RNA sequence by virtue of  compute-intensive algorithms called basecallers. This blog post presents the benchmarking results for two of those Oxford Nanopore basecallers — Guppy and Dorado — on AWS. This benchmarking project was conducted in collaboration between G42 Healthcare, Oxford Nanopore Technologies and AWS.

Building a 4x faster and scalable algorithm using AWS Batch for Amazon Logistics

Building a 4x faster and more scalable algorithm using AWS Batch for Amazon Logistics

In this post, AWS Professional Services highlights how they helped data scientists from Amazon Logistics rearchitect their algorithm for improving the efficiency of their supply-chain by making better planning decisions. Leveraging best practices for deploying scalable HPC applications on AWS, the teams saw a 4X improvement in run time.

Optimizing your AWS Batch architecture for scale with observability dashboards

AWS Batch customers often ask for guidance to optimize their architectures and make their workload to scale rapidly. Here we describe an observability solution that provides insights into your AWS Batch architectures and allows you to optimize them for scale and quickly identify potential throughput bottlenecks for jobs and instances.