How can I automatically start an AWS Glue job when a crawler run completes?
Last updated: 2020-03-17
I want to configure AWS Glue to automatically start a job when a crawler run completes. How can I do that?
Resolution
It's not possible to use AWS Glue triggers to start a job when a crawler run completes. Use one of the following methods instead:
- Create an AWS Lambda function and an Amazon CloudWatch Events rule. When you choose this option, the Lambda function is always on. It monitors the crawler regardless of where or when you start it. You can also modify this method to automate other AWS Glue functions. For more information, see How can I use a Lambda function to automatically start an AWS Glue job when a crawler run completes?
- Use AWS Glue workflows. This method requires you to start the crawler from the Workflows page on the AWS Glue console. For more information, see How can I use AWS Glue workflows to automatically start a job when a crawler run completes?
Did this article help you?
Anything we could improve?
Need more help?