Posted On: Nov 28, 2022
AWS Glue now offers custom visual transforms which let customers define, reuse, and share business-specific ETL logic among their teams. AWS Glue is a serverless, scalable data integration service that makes it easier to discover, prepare, move, and integrate data from multiple sources. With this new feature, data engineers can write reusable transforms for the AWS Glue visual job editor. Reusable transforms increase consistency between teams and help keep jobs up to date by minimizing duplicate effort and code.
You can define AWS Glue custom visual transforms using Apache Spark code as well as the user input form. You can also specify validations for the input form to help protect users from making mistakes. Once you save the files defining the transform to your AWS account, it automatically appears in the dropdown list of available transforms in the visual job editor. You can call custom visual transforms from both visual and code-based jobs, and sharing transforms between AWS accounts is straightforward.
To learn more, visit the AWS Glue documentation, read the blog post, and watch the launch video.
This feature is available in the same AWS Regions as AWS Glue.