Posted On: Jun 4, 2021
AWS Glue Studio now allows you to specify the settings for your streaming extract-transform-and-load (ETL) job in the visual job editor. This feature allows you to optimize your AWS Glue streaming ETL jobs for your use case. You can choose the window size for reading data from the data stream, whether to detect the schema of each record or use the schema from the AWS Glue Data Catalog, and connection settings that fine-tune how the AWS Glue job reads from the stream.
AWS Glue streaming ETL jobs continuously consume data from streaming sources, clean and transform the data in-flight, and make it available for analysis in second. AWS Glue Studio is a visual interface for AWS Glue that makes it easy to author, run, and monitor streaming ETL jobs.
This feature is available in the same AWS Regions as AWS Glue Studio: US East (N. Virginia, Ohio), US West (Northern California, Oregon), Africa (Cape Town), Asia Pacific (Hong Kong, Mumbai, Osaka, Seoul, Singapore, Sydney, Tokyo), Canada (Central), Europe (Frankfurt, Ireland, London, Milan, Paris, Stockholm), Middle East (Bahrain), and South America (Sao Paulo).
To learn more, visit our documentation for streaming ETL jobs and AWS Glue Studio.