Genomics Secondary Analysis Using AWS Step Functions and AWS Batch

The code creates four CloudFormation stacks in your AWS account including a setup stack to install the Guidance. The other stacks include a landing zone (zone) stack containing the common solution resources and artifacts, a deployment pipeline (pipe) stack defining the solution's CI/CD pipeline, and a codebase (code) stack providing the tooling, workflow definitions, and job execution environment source code.

The setup stack creates an AWS CodeBuild project containing the setup.sh script. This script creates the remaining CloudFormation stacks and provides the source code for both the AWS CodeCommit pipe repository and the code repository, once they have been created.

The landing zone (zone) stack stack creates the CodeCommit pipe repository, an Amazon CloudWatch event, and the AWS CodePipeline pipe pipeline which defines the continuous integration/continuous delivery (CI/CD) pipeline for the genomics workflow. The deployment pipeline (pipe) stack stack creates the CodeCommit code repository, an Amazon CloudWatch event, and the CodePipeline code pipeline.

The CodePipeline code pipeline deploys the codebase (code) CloudFormation stack. The resources deployed in your account include Amazon Simple Storage Service (Amazon S3) buckets, CodeCommit repositories for source code, AWS CodeBuild projects, AWS CodePipeline pipelines, Amazon Elastic Container Registry (Amazon ECR) image repositories, an example AWS Step Functions state machine, and AWS Batch compute environments, job queues, and job definitions. An example Amazon CloudWatch dashboard provides operational workload monitoring. In total, this solution enables building and deploying updates to both the genomics workflows, and the infrastructure that supports their execution.