Posted On: Mar 27, 2023
AWS Glue Crawler now supports enhanced crawl capabilities in AWS GovCloud (US) Regions, simplifying running Crawlers with AWS Lake Formation, discovering datasets, and populating schema in the AWS Glue Data Catalog.
With the enhanced crawl capabilities, you can use Lake Formation to simplify crawler setup and support centralized permissions for in-account and cross-account crawling of AWS S3 data lakes. You can reduce crawl times by using Amazon S3 Event Notifications on Glue Data Catalog tables, a capability that makes discovering datasets simpler by scanning only data based on events in Amazon S3. You can now crawl more data sources, including Snowflake, MongoDB Atlas, and native Delta lake tables, and govern with simplified auditing with cost metrics for past crawler executions.
AWS Glue Crawler enhanced crawl capabilities are already available in all commercial regions, China regions, and now in AWS GovCloud (US) Regions. To learn more about using Crawler with Lake Formation in data mesh architecture, read the blog, and visit the AWS Glue Crawler documentation.