Category: Best Practices

AWS Batch customers often ask for guidance to optimize their architectures and make their workload to scale rapidly. Here we describe an observability solution that provides insights into your AWS Batch architectures and allows you to optimize them for scale and quickly identify potential throughput bottlenecks for jobs and instances.

In this post, we will provide an overview of Numerical Weather Prediction (NWP) workloads, and the AWS HPC-optimized services for it. We’ll test three popular NWP codes: WRF, MPAS, and FV3GFS.

In this blog, we showcase the first version of Open Omics and benchmark three applications that are used in processing NGS data – sequence alignment tools BWA-MEM, minimap2, and single cell ATAC-Seq on Xeon-based Amazon Elastic Compute Cloud (Amazon EC2) Instances.

In this final part of this three-part blog series on building predictive models at scale in AWS, we will use the synthetic dataset and the models generated in the previous post to showcase the model updating and sensitivity analysis capabilities of the aws-do-pm framework.

In the first part of this three-part blog series, we introduced the aws-do-pm framework for building predictive models at scale in AWS. In this blog, we showcase a sample application for predicting the life of batteries in a fleet of electric vehicles, using the aws-do-pm framework.

Predictive models have powered the design and analysis of real-world systems such as jet engines, automobiles, and powerplants for decades. These models are used to provide insights on system performance and to run simulations, at a fraction of the cost compared to experiments with physical hardware. In this first post of three, we described the motivation and general architecture of the open-source aws-do-pm framework project for building predictive models at scale in AWS.

In this blog post, we help you understand the AWS Batch job termination process and how you may take actions to gracefully terminate a job by capturing SIGTERM signal inside the application. It provides you with an efficient way to exit your Batch jobs. You also get to know about how job timeouts occur, and how the retry operation works with both traditional AWS Batch jobs and array jobs.

This post covers the different ways you can encode a dependency between basic and array jobs in AWS Batch. We also cover why you may want to encode dependencies outside of Batch altogether using a workflow system like AWS Step Functions or Apache Airflow.

OpenFOAM is one the most widely used Computational Fluid Dynamics (CFD) packages and helps companies in a broad range of sectors (automotive, aerospace, energy, and life-sciences) to conduct research and design new products. In this post, we’ll discuss six practical things you can do as an OpenFOAM user to run your simulations faster and more cost effectively.

Many shared file systems are used in supporting read-intensive applications, like financial backtesting. These applications typically exploit copies of datasets whose authoritative copy resides somewhere else. For small datasets, in-memory databases and caching techniques can yield impressive results. However, low latency flash-based scalable shared file systems can provide both massive IOPs and bandwidth. They’re also easy to adopt because of their use of a file-level abstraction. In this post, I’ll share how to easily create and scale a shared, distributed POSIX compatible file system that performs at local NVMe speeds for files opened read-only.