AWS Compute Blog
Category: AWS Batch
Orchestrating high performance computing with AWS Step Functions and AWS Batch
This blog post describes several challenges common to orchestrating HPC workloads. I describe how Step Functions with AWS Batch can solve many of these challenges. I provide a project that contains several sample patterns and show how to deploy and test this in your account.
Read MoreManaging and Securing AWS Outposts Instances using AWS Systems Manager, Amazon Inspector, and Amazon GuardDuty
This post is written by Sumeeth Siriyur, Specialist Solutions Architect. AWS Outposts is a family of fully managed solutions that deliver AWS infrastructure and services to virtually any on-premises or edge location for a truly consistent hybrid experience. Outposts is ideal for workloads that need low latency access to on-premises applications or systems, local data […]
Read MoreMonitoring delay of AWS Batch jobs in transit before execution
This post is written by Nikhil Anand, Solutions Architect AWS Batch enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch processing jobs on AWS. With AWS Batch you no longer have to install and manage batch computing software or server clusters used to run your jobs. This lets you […]
Read MoreVisualizing AWS Step Functions workflows from the AWS Batch console
This post written by Dhiraj Mahapatro, Senior Specialist SA, Serverless. AWS Step Functions is a low-code visual workflow service used to orchestrate AWS services, automate business processes, and build serverless applications. Step Functions workflows manage failures, retries, parallelization, service integrations, and observability so builders can focus on business logic. AWS Batch is one of the […]
Read MoreBuilding well-architected serverless applications: Optimizing application performance – part 3
September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. This series of blog posts uses the AWS Well-Architected Tool with the Serverless Lens to help customers build and operate applications using best practices. In each post, I address the serverless-specific questions identified by the Serverless Lens along with the recommended best practices. See […]
Read MoreHow to authenticate private container registries using AWS Batch
This post was contributed by Clayton Thomas, Solutions Architect, AWS WW Public Sector SLG Govtech. Many AWS Batch users choose to store and consume their AWS Batch job container images on AWS using Amazon Elastic Container Registries (ECR). AWS Batch and Amazon Elastic Container Service (ECS) natively support pulling from Amazon ECR without any extra […]
Read MoreUsing shared memory for low-latency, intra-node communication in AWS Batch
In this post, I show how the new shared memory support in AWS Batch is able to improve performance while decreasing the latency of the intra-node communication. This performance gain can also lower the cost of running jobs overall.
Read MoreIntroducing Spot Blueprints, a template generator for frameworks like Kubernetes and Apache Spark
This post is authored by Deepthi Chelupati, Senior Product Manager for Amazon EC2 Spot Instances, and Chad Schmutzer, Principal Developer Advocate for Amazon EC2 Customers have been using EC2 Spot Instances to save money and scale workloads to new levels for over a decade. Launched in late 2009, Spot Instances are spare Amazon EC2 compute […]
Read MoreIntroducing retry strategies for AWS Batch
This post is contributed by Christian Kniep, Sr. Developer Advocate, HPC and AWS Batch. Scientists, researchers, and engineers are using AWS Batch to run workloads reliably at scale, and to offload the undifferentiated heavy lifting in their day-to-day work. But even with a slight chance of failure in the stack, the act of mitigating these […]
Read MoreHow to run 3D interactive applications with NICE DCV in AWS Batch
This post is contributed by Alberto Falzone, Consultant, HPC and Roberto Meda, Senior Consultant, HPC. High Performance Computing (HPC) workflows across industry verticals such as Design and Engineering, Oil and Gas, and Life Sciences often require GPU-based 3D/OpenGL rendering. Setting up drivers and applications for these types of workflows can require significant effort. Similar GPU […]
Read More