Posted On: Jul 19, 2022
Auto Scaling in AWS Glue Streaming ETL is now generally available. AWS Glue Streaming ETL jobs can now dynamically scale resources up and down based on the input stream. Auto Scaling helps customers reduce the cost and manual effort required to optimize resources by allocating the right resources necessary for Streaming ETL jobs.
AWS Glue Streaming ETL jobs continuously consume data from streaming sources, clean and transform the data in-flight, and make it available for analysis in seconds. AWS Glue Auto Scaling monitors each stage of the job run and turns workers off when they are idle or adds workers if additional parallel processing is possible. This helps customers reduce cost for their streaming jobs and the manual effort required for optimizing resources. For more details, please refer to the our documentation.