How do I use AWS Batch as a target for my CloudWatch Events rule?

Last updated: 2019-05-08

I want to use AWS Batch as a target for my Amazon CloudWatch Events rule. How can I do this?

Resolution

In this example, you configure an Amazon Simple Storage Service (Amazon S3) bucket to send an event to AWS Batch through a CloudWatch Events rule.

Get the ARN of your job queue and job definition

1.    Open the AWS Batch console.

2.    In the navigation pane, choose Job queues.

3.    Choose your job queue.

4.    In the Overview section, copy the Queue ARN, and then save it for later.

5.    In the navigation pane, choose Job definitions, and then choose your job definition.

6.    In the Job definitions attributes section, copy the Job definition ARN, and then save it for later.

Create the CloudWatch Events rule

1.    Open the CloudWatch console.

2.    In the navigation pane, choose Rules.

3.    Choose Create rule.

4.    In the Event Source section, choose Event Pattern.

5.    For Service Name, choose Simple Storage Service (S3).

6.    For Event Type, choose Object Level Operations.

7.    Choose Any operation.

8.    Choose Specific bucket(s) by name, and then enter the name of your bucket.

9.    In the Event Pattern Preview code editor, for the bucketName property, enter your bucket name.

10.    In the Targets section, choose Add target.

11.    From the main menu, choose Batch job queue.

12.    For Job queue, paste in the job queue ARN that you copied earlier.

13.    For Job definition, paste in the job definition ARN that you copied earlier.

14.    For Job name, enter a name for your AWS Batch job.

15.    In the Configure input section, choose Input Transformer.

16.    In the first input box, enter the S3 bucket values that will be sent when the event is triggered:

{"awsExampleBucketValue":"$.detail.requestParameters.bucketName","awsexampleS3KeyValue":"$.detail.requestParameters.key"}

17.    In the second input box, enter the Parameters structure that will be passed to the Batch job:

{"Parameters" : {"awsExampleBucket": awsExampleBucketValue, "awsExampleS3Key": awsexampleS3KeyValue}}

Note: Replace awsExampleBucketValue and awsexampleS3KeyValue with your own values.

18.    Choose either Create a new role for this specific resource or Use existing role.

Note: If you choose an existing role, that role should have a SubmitJob permission on it.

19.    Choose Configure details.

20.    For Name, enter the name for your rule.

21.    For State, select the Enabled checkbox.

22.    Choose Create rule.

Test the new rule that you created

1.    Open the Amazon S3 console.

2.    Choose your bucket.

3.    Choose Upload.

4.    Choose Add files, and then add a test file to the S3 bucket to trigger the CloudWatch Events rule that you created earlier.

Check your logs

1.    Open the AWS Batch console.

2.    In the navigation pane, choose Jobs.

3.    Choose your job with a Status of SUCCEEDED.

4.    On the Job details page, in the Attempts section, choose View logs. The log displays your bucket name and the object's key in the CloudWatch console.