AWS Architecture Blog

Category: AWS Batch

Workflow manager for genomics workflows

Genomics workflows, Part 3: automated workflow manager

Genomics workflows are high-performance computing workloads. Life-science research teams make use of various genomics workflows. With each invocation, they specify custom sets of data and processing steps, and translate them into commands. Furthermore, team members stay to monitor progress and troubleshoot errors, which can be cumbersome, non-differentiated, administrative work. In Part 3 of this series, […]

Solution overview for automating regenie workflows on AWS

Genomics workflows, Part 1: automated launches

Genomics workflows are high-performance computing workloads. Traditionally, they run on-premises with a collection of scripts. Scientists run and manage these workflows manually, which slows down the product development lifecycle. Scientists spend time to administer workflows and handle errors on a day-to-day basis. They also lack sufficient compute capacity on-premises. In Part 1 of this series, […]

Figure 1. Architecture diagram for autonomous driving simulation

How to Run Massively Scalable ADAS Simulation Workloads on CAEdge

This post was co-written by Hendrik Schoeneberg, Sr. Global Big Data Architect, The An Binh Nguyen, Product Owner for Cloud Simulation at Continental, Autonomous Mobility – Engineering Platform, Rumeshkrishnan Mohan, Global Big Data Architect, and Junjie Tang, Principal Consultant at AWS Professional Services. AV/ADAS simulations processing large-scale field sensor data such as radar, lidar, and […]

Figure 1. Managed File Transfer Process Flow

Managed File Transfer using AWS Transfer Family and Amazon S3

Financial, healthcare, retail and other companies exchange many different types of data. This can include stock information, healthcare claims, and sharing product data files with their partners. These companies need a managed file transfer solution that supports data transformation, and exchange of data over File Transfer Protocol over SSL (FTPS), and Secure File Transfer Protocol (SFTP). […]

CodeCommit. CodeBuild, ECR

Serverless Architecture for a Web Scraping Solution

If you are interested in serverless architecture, you may have read many contradictory articles and wonder if serverless architectures are cost effective or expensive. I would like to clear the air around the issue of effectiveness through an analysis of a web scraping solution. The use case is fairly simple: at certain times during the […]