Posted On: Nov 21, 2022
AWS Glue crawlers now support Snowflake, making it easier for you to understand updates to Snowflake schema and extract meaningful insights.
To crawl a Snowflake database, customers can create and schedule a Glue crawler with JDBC URL with credential information from AWS Secret Manager. A configuration option allows you to specify if you want the crawler to crawl the entire database or limit the tables by including schema/table path and exclude pattern to reduce crawl time. With each run of the crawler, the crawler inspects and catalogs information, such as updates or deletes to Snowflake tables, external tables, views, and materialized views in the AWS Glue Data Catalog. For Snowflake columns with non-Hive compatible types, such as geography or geometry, the crawler will extract that information as a raw data type and make it available in the Data Catalog.
AWS Glue crawler support for Snowflake is generally available in the following AWS Regions: US East (Ohio, N. Virginia), US West (Oregon), Asia Pacific (Tokyo, Osaka), Europe (Ireland). To learn more, read the blog, and visit the AWS Glue Crawler documentation to learn more.