Improving the Player Experience by Leveraging AWS Global Accelerator and Amazon GameLift FleetIQ

Building a game to serve worldwide users over the internet can be challenging. In this two-part series, we’ll walk you through how game developers improve the player experience worldwide in order to deploy more efficiently, achieve lower latency, improve in-game performance, and deliver game content faster.

In this first post, we focus on how to use AWS Global Accelerator to improve the performance for dynamic content, like gameplay and online store traffic. We also present a reference architecture for integrating AWS Global Accelerator with Amazon GameLift FleetIQ to easily deploy, operate, and scale cloud servers for multiplayer games. The second blog post, about using Amazon CloudFront for delivering static content such as game binaries and media assets, is coming soon.

What is AWS Global Accelerator?

AWS Global Accelerator is a networking service that improves your internet user performance by up to 60% leveraging the AWS global network infrastructure. Growing and maintaining your online gaming community requires a smooth and competitive gaming experience. Using AWS Global Accelerator enhances your players’ online experience by routing player traffic along the private AWS global network, reducing in-game latency, jitter, and packet loss. Use AWS Global Accelerator to accelerate traffic between players and your servers hosted in AWS Regions, and expand the global reach of your game. Acceleration is supported for both TCP and UDP-based protocols used by popular engines, such as Amazon Lumberyard, Unity, and Unreal Engine. AWS Global Accelerator edge locations route player traffic to game servers in any of the AWS Global Accelerator supported Regions, thus providing faster load times and a more consistent in-game experience. Handle your large traffic spikes, such as game launches and in-game events, without interrupting service on the highly available AWS global network.

Here are some reasons you may choose AWS Global Accelerator for your game:

Improve in-game performance – Your player traffic is routed through your accelerator and traverses the dedicated, congestion-free, redundant AWS global network instead of the public internet. Player traffic enters through the nearest AWS edge location, from where the request is routed through an optimized path towards the endpoint hosting your gaming session. These route and network optimizations will help keep your packet loss, jitter, and latency consistently low while improving performance by as much as 60%.

Reach more players without deploying additional infrastructure – Global Accelerator brings your players closer to the AWS Regions hosting your game servers by accelerating player traffic along the AWS global network. As a result, you can optimize deployment of your game fleet to fewer AWS Regions while still serving a global player population.

Integrate with your matchmaking service – For multi-player games, you can now use custom routing accelerators to deterministically route players, who are matched together based on factors such as geographic location, player skill, and gaming configuration, to a single session running on a game server. You can simply direct multiple players to a unique port on your accelerator from the matchmaking service. The accelerator maps the port to a specific Amazon EC2 destination private IP address and routes traffic there. This makes it easy to integrate AWS Global Accelerator with your matchmaking service by handing out the accelerator IP and port that players should connect to.

Simplify protecting your games from DDoS attacks – Global Accelerator helps mitigate the risk of malicious attacks. By reducing the number of networks your data passes through, and narrowing entry to two static IP addresses, you create a smaller virtual surface area for attack. Decreased surface area combined with default protection from AWS Shield decreases downtime and latency from Distributed Denial of Service (DDoS) attacks for your games hosted on AWS. You can also enable AWS Shield Advanced for automated resource-specific enhanced detection and mitigation, as well as 24×7 access to the AWS DDoS Response Team (DRT) for manual mitigations of sophisticated DDoS attacks.

What is Amazon GameLift FleetIQ?

Amazon GameLift is a service for multiplayer game session servers that aims to provide four main benefits:

• Low latency: Allow game servers to be deployed globally so they are as close to the players as possible to minimize latency.
• Flexibility: Allow game server groups to scale with the number of players, no matter how many or few games are running at that time.
• High availability: Maintain idle capacity so that new demand can be serviced right away, without waiting for a new server to start.
• Low costs: Make surplus capacity in the form of EC2 spot instances practical to use for game session servers, saving development studios a boat load of moolah. With GameLift, you can experience up to 70% cost-savings compared to existing on-premises deployments.

AWS released a feature to the service called GameLift FleetIQ that optimizes the use of low-cost EC2 Spot Instances for cloud-based game hosting with Amazon EC2. With GameLift FleetIQ, you can work directly with your hosting resources in Amazon EC2 and Auto Scaling while taking advantage of GameLift optimizations to deliver inexpensive, resilient game hosting for your players.

First, FleetIQ helps reduce server costs compared to on-premises and colocation servers. Second, it makes it easy for game studios with central tech teams to integrate FleetIQ with their games that are already launched. And third, you can migrate your game servers to FleetIQ temporarily for burst capacity or permanently.

Using AWS Global Accelerator with Amazon GameLift FleetIQ to Improve In-Game Performance

By leveraging AWS Global Accelerator custom routing accelerators to accelerate the traffic for your FleetIQ server groups, you can reduce in-game lag and jitter for players connecting to your game servers. Doing this puts game session traffic onto the fast fiber of the Amazon network at a network edge point of presence (POP). By reducing the number of network hops required to route latency-sensitive gameplay traffic for players, you can now provide players with a much better gaming experience. Additionally, your players can see reduced network jitter, which is a measurement of the variability in the latency of game traffic, due to fewer network hops and routing traffic through the AWS global network for a longer portion of its journey.

Using AWS Global Accelerator with GameLift FleetIQ is akin to running an express train along the same route as the local bus. The bus has lots of stops, which is great if you want to get to out-of-the-way places, but if you are traveling to the end of the line, you take the bus to the nearest express station and change to the train for a shorter total journey time.

The process of routing player traffic through your accelerator to game sessions running on your FleetIQ requires few steps to set up. First, let’s first look at the architecture to learn how to set this up with Amazon GameLift FleetIQ and AWS Global Accelerator. Second, we will walk you through how player traffic is accelerated through the accelerator to the game server hosting the game session.

Setting up Amazon GameLift FleetIQ servers and AWS Global Accelerator

In this section, we describe the set up for hosting your games servers in GameLift FleetIQ and integrating it with AWS Global Accelerator. The setup steps are denoted by the blue tags in the architecture diagram.

Step 1: GameLift FleetIQ game server group setup

During the normal course of operation, GameLift FleetIQ manages the servers in the Game Server Group. When you create a game server group (1), a new EC2 auto scaling group is created (2), and you can launch game servers (3) into the auto scaling group. FleetIQ controls the behavior of the auto scaling group (4).

The auto scaling group spans across multiple Availability Zones (5), and there is a subnet (6) in each with a routing table (7) to route traffic to the servers and back to the clients.

By managing the scaling function, FleetIQ is able to:

1. bring on enough server capacity in advance of the demand when increasing numbers of games are played; and
2. shut down those servers as quickly as possible when the demand subsides to ensure efficient utilization of EC2 spot instances.

Each Amazon EC2 instance has an IP address, and each game server (which is typically an OS process running on that instance) is listening on a port, specified by the server in the connection information supplied to FleetIQ when it registers itself. The game server determines an available port on the instance it is running on, then reserves it for itself by opening the port for listening. Then it registers the port with GameLift (10) so that GameLift can allocate servers to games. As soon as the game client knows the IP address and port, it could connect to the server directly, but we use the acceleration benefits provided by Amazon Global Accelerator’s custom routing accelerators. Note that for integration with AWS Global Accelerator, we recommend that you host game sessions on a pre-defined list of ports on the EC2 instances.

Step 2: Create a custom routing accelerator and register your EC2 Spot instances

In order to allow game traffic to be sent from the game client to the accelerator, first you create a custom routing accelerator (8) and add each subnet, that will contain your servers in the fleet, as endpoints to the custom routing accelerator. Make sure that your game servers are hosted on a pre-defined set of ports on each EC2 instance. You will configure these ports when you configure the endpoint groups on the accelerator (visit documentation). The custom routing accelerator will create a static mapping from each listener port on the accelerator to each IP address and the pre-defined port ranges in the subnets. For example, in the illustration below, port 6 on the custom routing accelerator is mapped to 10.0.16.0:26 while port while port 8 is mapped to 10.0.16.1:18. So when a player connects to port 6 on the accelerator, their traffic will be routed to 10.0.16.0:26.

During a scaling event, once you bring an EC2 spot instance up in a subnet, you can call the AllowCustomRoutingTraffic API to allow AWS Global Accelerator to route traffic to game servers in the EC2 instance. If an EC2 instance and port is not allowed to receive traffic, player traffic will be dropped by the accelerator at the edge. Similarly, before terminating an EC2 instance you can deregister the EC2 IP and ports by calling the DenyCustomRoutingTraffic API (9).

Now that you have set up AWS Global Accelerator with Amazon GameLift FleetIQ, let’s talk about how a player connects to your game server through the accelerator.

In-game traffic flow for players

In this section, we will walk you through how a player will authenticate themselves, be matched with other players by the matchmaking service and then eventually be routed through Global Accelerator to the game server that is hosting a multi-player game on the fleet. The steps are denoted by the pink tags in the architecture diagram above.

Step 1: Authentication

Let’s examine at how a player connects to the game client (or the installer/updater). When a player logs in, the game client already has an identity, generally a Steam ID, PlayStation Network ID, Apple ID, etc., depending on the platform the game is running on. The game logs into the identity provider (1), usually outside the game itself. The identity provider authenticates the game’s credentials and gives the game an identity token (2). The game (3) uses the AWS SDK to transmit the token to the game backend (4). This design uses Amazon Cognito, a managed identity service from AWS, to validate the token. Cognito retrieves the player information from the ID Token to establish their identity.

Once identified, Cognito requests temporary credentials for the player (5) from the Secure Token Service (STS), and returns them to the game. The game uses the credentials to assume a role which authorizes the game to use the AWS resources that it needs to access. In this case we only grant access to the matchmaker controller (which can have one of several designs, so it is represented here as a generic compute unit) and the Lambda function for checking matchmaking results.

Step 2: Matchmaking

The game uses the STS credentials to invoke the matchmaker controller with a request for a game (6). In this case, the matchmaker controller (7) ensures that the request is valid – for example, possibly checking that the player is not already in a game, or if the player hasn’t requested more than five games in the last fifteen minutes. This helps the system be more resilient to hacked clients.

Once it is satisfied that the request is genuine, the matchmaker controller finds the player a game by finding other compatible players that have made a request. The match is eventually determined.

The matchmaker controller claims a game session server (8) from the FleetIQ game server group (9). This is where the match will be hosted. FleetIQ returns the real address and port number of the server to the matchmaker controller. Because we are using AWS Global Accelerator, it is necessary to convert the IP address and port of the server to the accelerator IP address and port (10). The mapping created during endpoint registration can be  referenced by calling the ListCustomRoutingPortMappingsByDestination API. This API will return the accelerator port that is mapped to the server process that was claimed for this game session.

Step 3: Connect multiple players to the game to the server

The matchmaker controller could write a server-initiated message to a client connected via Websocket or HTTP/2, but in this design, we assume this is not implemented, and the results are stored in a DynamoDB table temporarily (11). The record in the temporary data table (12) is given a time to live (TTL) so that if it is never checked, it expires after a period of five or ten minutes. When the game thinks the results may be ready, and then every few seconds after that, the game makes a request (13) to invoke a Lambda function to see if their request has been processed. The Lambda function reads the DynamoDB table and finds the record that contains the matchmaking results (14). These results are passed back to the game client along with the AWS Global Accelerator IP address and port to connect to.

The player can now connect to the GameLift game server assigned to the game session using either of the accelerator IP addresses and port (15). AWS Global Accelerator (16) receives the request at the edge location (point of presence) nearest the player and uses the mapping to accelerate their traffic to the correct server. The players should all arrive pretty quickly (since they are all matched at the same time), and gameplay can start.

TCP or UDP traffic now flows freely from the players to the server (and back, since AGA accelerates traffic in both directions), allowing them to experience their game with the lowest possible latency achievable. This clearly results in happy players! Mission accomplished.

Stay tuned for part two of this series to learn about using Amazon CloudFront for delivering static content.