1.2 — We will use the standard batch console. If you see a page that says get started, click get started. Otherwise skip to step 2.

1.3 — Click skip wizard. We will use the standard Batch console for the purpose of this tutorial.

You will now see the AWS Batch console.

1.4 — A compute environment is a set of compute resources that AWS Batch will use to place your jobs. To create one, click on the compute environments section on the left-hand side of the AWS Batch console and then click on the create environment button to open the compute environment creation wizard.

1.5 — On the compute environment type, leave the default selection to managed. This will create a compute environment managed by AWS Batch that will be automatically scaled out and scaled in as batch jobs are scheduled and compute capacity is needed.

Then, set up a compute environment name e.g. spot-compute-environment.

On the service role drop-down, select AWSBatchServiceRole if it exists on your account, otherwise select create new role. This is an IAM role that is used by AWS Batch to manage resources on your behalf.

On the instance role, select ecsInstanceRole IAM role if it exists on your account, otherwise select create new role to create it. This is an IAM role that will be attached to EC2 instances to make AWS API calls on your behalf (AWS Batch uses Amazon ECS to create the compute environment).

Then, on the EC2 key pair section, optionally select an EC2 key pair if you want to be able to SSH to the EC2 instances that AWS Batch will create.

1.6 — Now, scroll down and go to the configure your compute resources section.

Select Spot as provisioning model, leave maximum price blank (this will use the maximum price of the instances set to its On-Demand price, but you will pay always the current price).

Leave the default values for allowed instance types (optimal) and allocation strategy (SPOT_CAPACITY_OPTIMIZED). These settings will make AWS Batch pick the best instance types for your jobs from the m4, r4 and c4 instance families and also from the deepest Spot capacity pools, reducing the chances of Spot Instance interruptions to happen while your jobs are running (as interruptions happen when EC2 needs to claim capacity for a spike of demand of on-demand instances). You can add additional instance families or specific types, including GPU instance families (P2 and P3) if your jobs require it; AWS Batch will pick the optimal set of instances based on your jobs’ requirements, while benefiting from the 90% discount Spot Instances offer over On-Demand pricing.

1.7 — Leave the rest of the values on the configure your compute resources as default. Then, scroll-down to the networking section. By default, AWS Batch will select your Default VPC. If your account does not have a default VPC or you want to deploy your Batch compute environment on a specific VPC, select it here, as well as the subnets to use. If you are not using the default VPC, make sure your VPC configuration conforms with the networking requirements specified here.

Scroll-down to the very bottom of the page, and click on the create button to create the compute environment.

1.8 — Your newly created compute environment will show up on the console.

2.2 — Set up a queue name (e.g. my-batch-job-queue), a priority of 10 and on the connected compute environments for this queue select the spot-compute-environment from the list.

Then click on the create job queue button.

3.2 — On the job definition name give your job a name (e.g. my-first-batch-job). Job attempts define the number of times AWS Batch will try to execute your job. Although in average only less than 5% of Spot Instances are interrupted, interruptions can occur if EC2 needs the capacity back. To maximize uptime, set the job attempts value to 2 or higher so AWS Batch retries the execution if the instance where the job runs gets interrupted or fails. Optionally, define also an execution timeout value.

3.3 — Leave the parameters section blank (feel free to read more about Job definition parameters here).

Then, on the environment section leave job role blank (you can use this parameter to assign an IAM role to your job, so your job can invoke API calls to AWS services); set up amazonlinux as container image and on the command section, click on the JSON tab and paste the following into the text box: ["sh", "-c", "echo \"Hello world from job $AWS_BATCH_JOB_ID \". This is run attempt $AWS_BATCH_JOB_ATTEMPT"]. You will assign 1 vCPU and 256 MB of memory to this job.

This sample job will run on an amazonlinux container image and will print a “Hello world” message including its job ID assigned by AWS Batch and the execution attempt number. This data is provided by AWS Batch in the form of environment variables, and you can use it on your job. Additionally, you can pass additional environment variables to your jobs. To learn more about this feature, take a look at our documentation here.

3.4 — Leave the rest of the settings by default, and scroll-down to the very bottom to click on the create job definition button.

4.2 — On the console, give a job name to your job (e.g. first-batch-job-run), select my-first-batch-job on job definition and leave the queue we created before on the job queue. Leave the rest of the settings as default, and then scroll-down to the very bottom to click on the submit job button.

4.3 — At this stage, your batch job has been placed on the queue and AWS Batch will take care of scaling compute resources on the compute environment linked to the queue; in our case selecting the most appropriate instance type to run our job from the deepest Spot capacity pool.

Go to the dashboard and keep refreshing the job queues section, so you see the job transitioning through the different states until it ends up on SUCCEEDED. If you refresh the compute environments section you will also see how Batch increases the Desired vCPUs to run our job. (NOTE: It will take ~2 minutes for AWS Batch to spin up the compute resources and run your job. Feel free to navigate through the console and get familiar with it).  

4.4 — Once your job ends up SUCCEEDED click on the number below SUCCEEDED. You will be redirected to the jobs section of the console.

4.5 — Click on the job id to open its details.

4.6 — Scroll down a bit until you will find the attempts section where you have a link to view logs for your job.

4.7 — Click on the view logs link to get redirected to Amazon CloudWatch Logs to see the logs for your job. A new window will open and you should see an output similar to the following with the logs of your job.

6.1 — Go to the job queues section of the AWS Batch console and select your queue. Then, click on the disable button.

Click on the circle arrow button to refresh the console and wait for a minute until the queue state is DISABLED (Status should be VALID); then go ahead and click delete to delete it. Wait until the queue is fully deleted (keep refreshing with the circular arrow button until the queue disappears from the console).

6.2 — Go to compute environments and select your compute environment, then click on disable. Keep refreshing the console using the circle arrow button and allow for some for AWS Batch to disable the compute environment. Status should be VALID and State should be DISABLED.

Then, select the compute environment and click on the delete button to delete the environment.