AWS DevOps Blog

Category: AWS Glue

Figure 1 Solution Overview

How to unit test and deploy AWS Glue jobs using AWS CodePipeline

This post is intended to assist users in understanding and replicating a method to unit test Python-based ETL Glue Jobs, using the PyTest Framework in AWS CodePipeline. In the current practice, several options exist for unit testing Python scripts for Glue jobs in a local environment. Although a local development environment may be set up […]

architecture diagram showing ETL process

Using AWS DevOps Tools to model and provision AWS Glue workflows

This post provides a step-by-step guide on how to model and provision AWS Glue workflows utilizing a DevOps principle known as infrastructure as code (IaC) that emphasizes the use of templates, source control, and automation. The cloud resources in this solution are defined within AWS CloudFormation templates and provisioned with automation features provided by AWS […]