Creating a Build Environment on AWS with Incredibuild

Game developers often need to compile large amounts of C++ code, which requires lots of CPU resources, and developers also need to process other types of heavy compute tasks such as; shader compilation, rendering, asset creation, image conversion, lightmap baking and more. These tasks can negatively impact the continuous development experience, and productivity, by occupying the local machine resources and studio build farms. We can alleviate these production bottlenecks, and scale on demand, by creating a distributed build environment in the cloud. These compute intense processes can then be delegated to the cloud build environment and, more importantly, can be accelerated. So, how can we do this and with what type of solution?

Overview of solution

Incredibuild for Cloud Optimization on the AWS Marketplace

Incredibuild is a scalable software solution for game developers to accelerate CI build and compilation times, providing a powerful method for continuously building, improving, and releasing games. Incredibuild automatically monitors the amount of compute cores needed at any given moment to dynamically allocate, or de-allocate, Amazon EC2 Spot or On-Demand Instances based on your predefined machine type, and budget preference, in order to meet production demands.

Incredibuild Cloud enables you to use cloud Virtual Machines (VMs) to increase the number of cores you want to use for executing build tasks, without the need to manually install or maintain any on-premises resources. To use Incredibuild Cloud all you need is the install package and an active AWS account. Incredibuild Cloud is now available here on the AWS Marketplace and will automatically launch and terminate VMs according to your on-going needs. The Incredibuild Coordinator Monitor displays the current activity of your resources on AWS, and the solution will not only accelerate your development process, but also save you money on AWS usage.

In this blog post, we will show you how to setup Incredibuild on AWS.

First Step

If you simply want to quickly deploy the new Incredibuild Coordinator Monitor in AWS, you can be up and running in minutes with the Incredibuild AMI from the AWS Marketplace.

Let’s start by launching an Amazon EC2 Instance from the AWS Marketplace.

Press “Try for free” to create a Contact and Accept.  See pricing details and examples in the Pricing Information section on the Incredibuild AWS Marketplace page for paid subscription costs.

Click “Set up your account” to start the setup of Incredibuild Cloud.

If you do not have an account, create a new one here.

In the next step, you need to specify the IAM Role required for AWS operation.

If there isn’t appropriate IAM role, create it here.

First, open the AWS IAM Management Console, select “Policies” section, and click the “Create Policy”.

Select the “JSON” tab and paste the required text. The contents are as follows in the image below.

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"servicequotas:ListServices",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DeleteSubnet",
"ec2:DescribeInstances",
"ec2:RequestSpotInstances",
"ec2:CreateVpc",
"ec2:RequestSpotFleet",
"ec2:AttachInternetGateway",
"ec2:DescribeSpotInstanceRequests",
"servicequotas:GetServiceQuota",
"ec2:ModifySubnetAttribute",
"ec2:DescribeInternetGateways",
"ec2:ModifySpotFleetRequest",
"ec2:DescribeNetworkInterfaces",
"ec2:StartInstances",
"ec2:CreateRoute",
"ec2:CreateInternetGateway",
"ec2:CreateSecurityGroup",
"ec2:DeleteInternetGateway",
"ec2:ModifyInstanceAttribute",
"ec2:DescribeRouteTables",
"servicequotas:ListServiceQuotas",
"ec2:DescribeCapacityReservations",
"ec2:TerminateInstances",
"ec2:DescribeLaunchTemplates",
"ec2:CreateTags",
"ec2:RunInstances",
"ec2:CreateFleet",
"ec2:DetachInternetGateway",
"ec2:StopInstances",
"ec2:DescribeSecurityGroups",
"ec2:CreateLaunchTemplateVersion",
"ec2:RevokeSecurityGroupIngress",
"ec2:DescribeImages",
"ec2:CancelSpotInstanceRequests",
"ec2:CreateLaunchTemplate",
"ec2:DescribeSecurityGroupRules",
"ec2:DescribeVpcs",
"ec2:DeleteSecurityGroup",
"ec2:DeleteVpc",
"ec2:CreateSubnet",
"ec2:DescribeSubnets"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/*",
"Condition": {
"StringLike": {
"iam:AWSServiceName": "spot.amazonaws.com"
}
}
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "arn:aws:iam::*:role/aws-service-role/ec2fleet.amazonaws.com/*",
"Condition": {
"StringLike": {
"iam:AWSServiceName": "ec2fleet.amazonaws.com"
}
}
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"iam:AttachRolePolicy",
"iam:PutRolePolicy"
],
"Resource": "arn:aws:iam::*:role/aws-service-role/ec2fleet.amazonaws.com/*"
}
]
}

On the next screen, enter a policy name and save it.

Next, select “Roles” section, and click the “Create role”.

Specify “AWS Service” for Trusted entity type and “EC2” for Use case.

On the next screen, specify the policy you created.

On the next screen, enter a role name and save it.

Finally, open the Role you just created, select the “Trust Relationships” tab, and paste the required text.

The contents are as follows in the image below, and rewrite <Your_External_ID> to any value.

Write down the “ARN“ and “ExternalId“ shown in the image.

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::827268715074:user/incrediCloud"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "<Your_External_ID>"
}
}
}
]
}

This completes the creation of IAM Role.

For more information on these IAM policies and roles, please refer to the official Incredibuild Documentation.

Return to the User registration screen, enter “Role ARN” and “External ID” and click “Next”.

Next, specify the region/VPC where the Helper Agent is to be deployed.

“Cloud Region” should be chosen near the users of this service. If you want to deploy Amazon EC2 Instance in an existing VPC, choose “Use your existing network”. Otherwise you choose “Let Incredibuild create a network for you”.

Finally, configure the Amazon EC2 Instance Type you wish to designate as a Helper Agent. At first, there is no problem with the default settings.

Press “Enable Cloud” on the last confirmation screen.

If the setup is successful, the following screen will appear.

Now let’s check that the Agent can connect to the activated Coordinator.

Start an Amazon EC2 Windows Instance for confirmation. For the VPC to launch EC2, select the one created or specified in the Coordinator setup.

You can use AWS Systems Manager Fleet Manager to connect to EC2.

Install the Agent of Incredibuild here.

The Agent installer can be obtained from the e-mail sent upon completion of setup.

After the installation is completed, check the Coordinator Manager and you will see that the instance has been added as an Initiator Agent.

If it isn’t added, communication between the Agent and the Coordinator may not be working.

Please check the EC2 Security Groups of the Agent and the Coordinator, and make sure that “Coordinator Ports” in the Setup screen is allowed.

Now the Coordinator setup and check it works properly are complete.

Operating the build environment

If you are in a position to manage your infrastructure, it is important to optimize your AWS usage and cost.

This section describes how to optimize by scaling in/out Helper Agents using Incredibuild Cloud.

The second half of the article describes the network architecture for secure use of Incredibuild Cloud.

About Incredibuild Architecture

In Incredibuild, a build job is requested by the Initiator Agent, and the job is executed by Helper Agents.

Incredibuild Architecture and Terminology documentation

We’ve already started the Initiator Agent in the previous section to confirm the Coordinator works.

From here, let’s check how to add the Helper Agent.

Add Helper Agents using Incredibuild Cloud

Helper Agents need to be added to process jobs requested by the Initiator, and with Incredibuild Cloud you can automatically scale in/out Helper Agents as required.

This configuration is done from the “Resource Management” screen that appeared in First Step. Let’s check the screen again.

Here it is possible to specify the Amazon EC2 Instance Type, Amazon Elastic Block Store type, and maximum number of scale-out units to be specified for the Helper Agent.

Since the use of these Helper Agents accounts for a large portion of the cost, it is necessary to make the appropriate settings here.

In addition, from “Fallback Machine Types” at the bottom of the screen, you can set up to 9 alternative types in case the Default Instance Type cannot be started, as shown in the following screen.

When the configuration is complete, the number of 7 Helper Agents will be started as shown on the Amazon EC2 management console.

When the activation is complete, the Agents are also added to the Coordinator Manager.

By choosing Amazon EC2 Spot Instances, you can take advantage of unused EC2 capacity in the AWS cloud.

Spot Instances are available at up to a 90% discount compared to On-Demand prices.

When Amazon EC2 needs the capacity back, Spot Instances are terminated or stopped, so running jobs may be interrupted.

However, interrupted jobs are re-executed by the Coordinator, so users can benefit from low prices without worrying about job failures.

You can see that the instance started with “Spot Instance” enabled is correctly used, with the Lifecycle set to “spot” as shown in the following screen.

It is also possible to stop instances that are not used for a certain period of time by setting “VM Idle Timeout”.

Check the instances that have been started from the AWS Management Console.

As you can see on the screen, instances other than the Coordinator are automatically stopped by the Incredibuild Cloud if they remain idle for a certain period of time.

In this way, it is possible to control the optimal use of instances only from the settings screen of Incredibuild.

For more detailed instructions on Incredibuild Cloud, please refer to the official documentation.

Architecture

An architectural diagram for using Incredibuild Cloud from the office or home is shown below.

Possible use cases include (1) from the office, (2) from home, and (3) from workstations set up on AWS.

Incredibuild establishes secure communication between each Incredibuild node (Coordinator, Initiator, Helper).

To further increase security, it is possible to set up a leased line or VPN between AWS and the user’s PC.

AWS Direct Connect or AWS Site-to-Site VPN can be used between the office and AWS, and AWS Client VPN can be used between home and AWS.

By deploying virtual workstations within an Amazon Virtual Private Cloud (VPC) you can establish a more secure connection with Incredibuild by keeping the communication within the same VPC. High-performance remote display protocols like NICE DCV from Amazon, Teradici, or Parsec, can be used to stream the user interface to your simpler client machines.

Cleaning Up

To avoid incurring future charges, you should terminate EC2 instances you started on the Amazon EC2 management console.

Conclusion

Now you can deploy Incredibuild Coordinator to AWS in minutes using the Incredibuild AMI on AWS Marketplace. Check out this AWS Sample to deploy a more advanced Cloud Game Development environment including setting up a virtual workstation and version control.

By running Incredibuild on AWS, developers can build with flexibility, scalability and security in mind. Make games faster, and more efficiently, by creating a secure distributed development environment that is not restricted to any one location.

Incredibuild is one of many curated solutions for Cloud Game Development published here in the AWS Solution Library, and for more information about Cloud Game Development please visit landing page at AWS for Games .

To purchase a license of Incredibuild, please contact Incredibuild via the “Support Information” link in Marketplace.

This Blog was written by Yoshihiro Nagata, Senior Specialist Solution Architect, and Andy Payne, Head of Solutions, Game Production in the Cloud.