Networking & Content Delivery

Introducing AWS Global Accelerator custom routing accelerators

AWS Global Accelerator now supports custom routing accelerators, a new type of accelerator that allows you to use your own application logic to route multiple users to a specific Amazon EC2 destination, while still leveraging the benefits of Global Accelerator.

Global Accelerator is a networking service that improves your internet user performance and availability by using the AWS global network infrastructure. Global Accelerator’s automatic routing optimizations help keep your packet loss, jitter, and latency consistently low. Today, standard accelerators automatically route traffic to a healthy endpoint that is nearest to your user. This works for a wide variety of use cases such as A/B testing, blue-green deployments, API acceleration, and live video ingest.

Since standard accelerators are designed to load balance traffic, you cannot deterministically route multiple users to a specific EC2 destination behind your accelerator, as is required for some use cases. One example is multi-player gaming applications where you want to assign multiple players to a single session on a game server based on factors such as geographic location, player skill, and gaming configuration. Another example is VoIP, EdTech and social media applications that assign multiple users to a specific media server to initiate voice, video, and messaging sessions.

With a custom routing accelerator, you can direct multiple users to a unique port on your accelerator. The accelerator will map the port to a specific EC2 destination private IP address and port in single or multiple AWS Regions and will route traffic there. This feature makes it easier to integrate Global Accelerator with your application logic, such as matchmaking servers or session border controllers (network elements that protect and regulate IP traffic flows for real-time communication workflows). With custom routing accelerators, you can now leverage AWS Global Accelerator as the single point of entry for your application while deterministically sending your user traffic to specific EC2 destinations in any AWS Region.

In this blog post, we’ll walk you through how custom routing accelerators work and provide step-by-step guidance for setting up a new accelerator.

How does custom routing work?

To better understand how custom routing works, let’s look at an example where you have multiple EC2 instances in one Region, and where you run your application on ports 80 and 81. You currently use custom application logic to match users to a session running on an EC2 instance at a specific port. For example, in the following diagram, the users in blue connect to port 81 on Instance A.

The following diagram shows how custom application logic matches internet users to a port on a specific EC2 instance:

One way to match users is to create an Elastic IP address for each of the instances. But doing that is cumbersome and also exposes your EC2 instances to the internet, which could pose security threats. You can mitigate this risk and improve performance by adding a standard accelerator. But if you do that, you lose the deterministic routing functionality that you need.

Now, with this Global Accelerator feature launch, you can use a custom routing accelerator to deterministically route multiple users to a specific EC2 instance and port, such as Instance A in our example. A custom routing accelerator automatically creates a static mapping from the ports on your accelerator to the corresponding EC2 destinations. Users can connect to either of the two anycast IP addresses allocated to your accelerator. Your custom application logic can retrieve this auto-generated mapping to match multiple users to the correct session. Global Accelerator routes all accelerator traffic over the AWS global network backbone to the appropriate destination. The following diagram shows the port mapping for a custom routing accelerator, and illustrates how users are routed to the correct instance and port.

Now let’s look at how to create and configure a custom routing accelerator. For simplicity, we’ll assume that you have four EC2 instances running your application on ports 80 and 81 in a /28 VPC subnet in a single Region.

Create and configure a custom routing accelerator

To illustrate how you can set up a custom routing accelerator, we’ll walk through the steps using the AWS Command Line Interface (CLI). For a step-by-step guide using the AWS Management Console, see Getting Started with AWS Global Accelerator custom routing in the AWS Global Accelerator Developer Guide.

Initial setup

You’ll need some resources to work with when you create the custom routing accelerator. So before we get started, create a subnet in a VPC, and then launch four EC2 instances with a web server on each one that listens on ports 80 and 81 by doing the following:

  1. In the AWS Management Console, in the VPC console, create a test subnet in a VPC in an AWS Region of your choice. Make the subnet /28 and take note of the subnet ID; you’ll need the ID later when you add the VPC to your accelerator. For our example setup, we created the subnet in the us-east-1 Region. For more information, see Working with VPCs and subnets in the documentation.
  2. In the subnet that you just created, launch four EC2 instances, and set up Apache or Nginx web servers on them. For our example, we launched four EC2 instances in the subnet. An easy way to set this up is to use EC2 user data at launch (see the Advanced Details tab in the Management console). For more information, including a sample script for Apache web server, see Running commands on your Linux instance at launch in the documentation.

Make sure that the Security Group and Network ACLs associated with your EC2 instances allow traffic on the ports that you want to use in the endpoint groups. This example uses TCP ports 80 and 81.

Now you’re ready to create the accelerator.

Step 1. Create a custom routing accelerator

To create a custom routing accelerator called demoAccelerator, run the following CLI command. The command and the response look like the following:

$ aws globalaccelerator create-custom-routing-accelerator --name demoAccelerator
{
    "Accelerator": {
        "AcceleratorArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917",
        "Name": "demoAccelerator",
        "IpAddressType": "IPV4",
        "Enabled": true,
        "IpSets": [
            {
                "IpFamily": "IPv4",
                "IpAddresses": [
                    "1.2.3.4",
                    "5.6.7.8"
                ]
            }
        ],
        "DnsName": " a1b2c3d4e5f6g7h8i.awsglobalaccelerator.com",
        "Status": "IN_PROGRESS",
        "CreatedTime": "2020-10-28T10:56:28-05:00",
        "LastModifiedTime": "2020-10-28T10:56:29-05:00"
    }
}

Step 2. Create a listener

Next, create a listener for your custom routing accelerator. Listeners process inbound connections from users, based on the ports that you specify. Run the following CLI command to create a listener for your accelerator that processes connections on ports 80 to 200. Each of these destination ports on the listener will have a unique mapping to a specific EC2 destination within the subnet endpoints that you’ll specify for the accelerator. Note that the custom routing accelerator auto-generates the mapping from a listener port to the EC2 destination. This mapping allows you to route your users to a maximum of 121 unique destination EC2 instances and ports.

The command and the response to create the listener look like the following:

$ aws globalaccelerator create-custom-routing-listener --accelerator-arn arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917 --port-ranges FromPort=80,ToPort=200
{
    "Listener": {
        "ListenerArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1",
        "PortRanges": [
            {
                "FromPort": 80,
                "ToPort": 200
            }
        ]
    }
}

Step 3. Create an endpoint group

Endpoint groups in custom routing accelerators route requests to one or more VPC subnets in a single Region. The VPC subnets contain the instances that you want to associate with your accelerator. For each endpoint group, you specify a port range and protocol that you want to direct traffic to on your destination EC2 instances. The listener ports you created in step 2 will be automatically mapped to the destination ports defined in your endpoint groups.

In our example, we want the destination EC2 instances to accept both TCP and UDP traffic on only ports 80 and 81. So we configure our endpoint group with a port range of 80-81 and specify both protocols.

The command and the response look like the following:

$ aws globalaccelerator create-custom-routing-endpoint-group --listener-arn arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1 --endpoint-group-region us-east-1  --destination-configurations FromPort=80,ToPort=81,Protocols=TCP,UDP
{
    "EndpointGroup": {
        "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002",
        "EndpointGroupRegion": "us-east-1",
        "DestinationDescriptions": [
            {
                "FromPort": 80,
                "ToPort": 81,
                "Protocols": [
                    "TCP",
                    "UDP"
                ]
            }
        ],
        "EndpointDescriptions": []
    }
}

Step 4. Add endpoints to the custom routing endpoint group

The next step is to add VPC subnet endpoints to the endpoint group so your users can be routed to EC2 instances that run in the subnets. The VPC subnet endpoints contain the IP addresses of the destination EC2 instances that host your application. Note that custom routing accelerators support only VPC subnets as endpoints and route traffic only to EC2 instances within those subnets.

Run the following command to add your VPC subnet as an endpoint, using the subnet ID that you noted earlier. In our example, the subnet ID is subnet-0a08c428fc7d7af6e:

$ aws globalaccelerator add-custom-routing-endpoints --endpoint-group-arn arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002 --endpoint-configurations EndpointId=subnet-0a08c428fc7d7af6e
{
    "EndpointDescriptions": [
        {
            "EndpointId": "subnet-0a08c428fc7d7af6e"
        }
    ],
    "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002"
}

You might encounter the following error when you run the command:

“An error occurred (LimitExceededException) when calling the AddCustomRoutingEndpoints operation: There are not enough accelerator ports remaining to support the requested endpoints.”

This is because the number of listener ports configured in step-2 is lesser than the number of unique destination EC2 IP addresses and ports. Try increasing the number of listener ports for your accelerator, or use a VPC subnet with a smaller CIDR range.

Step 5. Allow specific IP addresses and ports to receive traffic

With a custom routing accelerator, you must enable IP addresses and ports in a VPC to receive traffic. By default, all IP address and port destinations in a subnet are not allowed to receive traffic in Global Accelerator. This prevents your entire subnet from being exposed to the public internet. Unless you enable the destinations, all traffic to the destination EC2 IP addresses and ports in the subnet is dropped at the edge.

In our example, we have four EC2 instances with the following private addresses: 172.31.100.5, 172.31.100.10, 172.31.100.14 and 172.31.100.18. We’ll choose to enable three of these IP addresses and ports to receive traffic in the subnet, by running the following command. This will enable traffic to all of the private IP addresses except 172.31.100.10:

$ aws globalaccelerator allow-custom-routing-traffic --endpoint-group-arn arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002  --endpoint-id subnet-0a08c428fc7d7af6e --destination-addresses 172.31.100.5 172.31.100.14 172.31.100.18 --destination-ports 80 81

Note that your custom application logic can also call these APIs to allow or deny traffic to specific destinations. For example, you might want to scale up the destinations that can receive traffic in preparation for an event. As you add new EC2 instances to each subnet endpoint, you can use your custom application logic to allow traffic to those destinations.

Step 6. List the custom routing port mapping

When Global Accelerator creates your custom routing accelerator, it auto-generates a static mapping from the ports to every possible destination private IP and port in your subnet(s), whether they are allowing traffic or not. You can retrieve the port mapping to route user traffic to specific EC2 destinations. Since the mapping does not change after you create the accelerator, your custom logic can run a command to list the mapping, and then save the result. You can use the mapping in real time to direct users to the correct destination for a session.

The following command lists all of your port mappings:

$ aws globalaccelerator list-custom-routing-port-mappings --accelerator-arn arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917
    "PortMappings": [
        {
            "AcceleratorPort": 82,
            "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002",
            "EndpointId": "subnet-0a08c428fc7d7af6e",
            "DestinationSocketAddress": {
                "IpAddress": "172.31.100.5",
                "Port": 80
            },
            "Protocols": [
                "TCP",
                "UDP"
            ],
            "DestinationTrafficState":"ALLOW"
        },
        {
            "AcceleratorPort": 83,
            "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002",
            "EndpointId": "subnet-0a08c428fc7d7af6e",
            "DestinationSocketAddress": {
                "IpAddress": "172.31.100.5",
                "Port": 81
            },
            "Protocols": [
                "TCP",
                "UDP"
            ] ,
            "DestinationTrafficState":"ALLOW"
        },
        {
            "AcceleratorPort": 92,
            "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002",
            "EndpointId": "subnet-0a08c428fc7d7af6e",
            "DestinationSocketAddress": {
                "IpAddress": "172.31.100.10",
                "Port": 80
            },
            "Protocols": [
                "TCP",
                "UDP"
            ] ,
            "DestinationTrafficState":"DENY"
        },
        {
            "AcceleratorPort": 93,
            "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002",
            "EndpointId": "subnet-0a08c428fc7d7af6e",
            "DestinationSocketAddress": {
                "IpAddress": "172.31.100.10",
                "Port": 81
            },
            "Protocols": [
                "TCP",
                "UDP"
            ] ,
            "DestinationTrafficState":"DENY"
        },
        {
            "AcceleratorPort": 116,
            "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002",
…
…
            "EndpointId": "subnet-0a08c428fc7d7af6e",
            "DestinationSocketAddress": {
                "IpAddress": "172.31.100.14",
                "Port": 80
            },
            "Protocols": [
                "TCP",
                "UDP"
            ]
        },
		{
            "AcceleratorPort": 117,
            "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002",
            "EndpointId": "subnet-0a08c428fc7d7af6e",
            "DestinationSocketAddress": {
                "IpAddress": "172.31.100.14",
                "Port": 81
            },
            "Protocols": [
                "TCP",
                "UDP"
            ]
        },
		{
            "AcceleratorPort": 120,
            "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002",
            "EndpointId": "subnet-0a08c428fc7d7af6e",
            "DestinationSocketAddress": {
                "IpAddress": "172.31.100.18",
                "Port": 80
            },
            "Protocols": [
                "TCP",
                "UDP"
            ]
        },
		{
            "AcceleratorPort": 121,
            "EndpointGroupArn": "arn:aws:globalaccelerator::123456789012:accelerator/e0f0ecfa-85f2-4d9e-b0ba-adac33e32917/listener/24b106b1/endpoint-group/355bec295002",
            "EndpointId": "subnet-0a08c428fc7d7af6e",
            "DestinationSocketAddress": {
                "IpAddress": "172.31.100.18",
                "Port": 81
            },
            "Protocols": [
                "TCP",
                "UDP"
            ]
        }
    ]
}

Test the custom routing accelerator

We’ve set up our custom routing accelerator with four destination EC2 instances (172.31.100.5, 172.31.100.10, 172.31.100.14, and 172.31.100.18) in one /28 subnet endpoint, and enabled traffic to three of the destinations. The following diagram illustrates how five users are routed through Global Accelerator to the EC2 instances in the VPC subnet endpoint. The custom routing port mapping information shown in the diagram comes from the response returned when you run the ListCustomRoutingPortMappings API call, as shown earlier in Step 6 of the accelerator setup.

The diagram shows that each user, except the last, is routed to a specific IP address and port that is allowed to receive traffic in the VPC subnet. The last user is trying to connect to a destination that you haven’t allowed to receive traffic. That user’s traffic is dropped at the edge, which protects destinations in the subnet that should not receive traffic.

You can use a Curl command to see the contents of the index pages on each the EC2 instances that can receive user traffic. In our example, we configured the EC2 instances to use instance metadata to return the private IP address, the AWS Region, and the Availability Zone, as shown in the following:

$ curl http://a1b2c3d4e5f6g7h8i.awsglobalaccelerator.com:82
   Served by 172.31.100.5 in us-east-1 Region (us-east-1d AZ)
$ curl http://a1b2c3d4e5f6g7h8i.awsglobalaccelerator.com:116
   Served by 172.31.100.14 in us-east-1 Region (us-east-1d AZ)
$ curl http://a1b2c3d4e5f6g7h8i.awsglobalaccelerator.com:117
   Served by 172.31.100.14 in us-east-1 Region (us-east-1d AZ)
$ curl http://a1b2c3d4e5f6g7h8i.awsglobalaccelerator.com:92
   User connection will timeout, since their traffic is dropped at the edge

Integrate a custom routing accelerator into your workflow

Now let’s see how you could integrate a custom routing accelerator into your workflow. The following diagram shows a simplified example workflow:

In this example, custom application logic, such as a matchmaker, matches users that you want to route to a specific destination EC2 IP and port. The application logic uses an API call, ListCustomRoutingPortMappings, to retrieve and store the preconfigured static mapping between the accelerator IP addresses and ports and the destination IP addresses and ports. This enables any user that connects to the application to be routed through Global Accelerator to the correct destination EC2 instance where the session is running.

The workflow goes like this:

  1. A user connects to the application.
  2. The custom application logic matches the user with other users that should be routed to the same destination by first doing the following:
    • It checks to find destinations that are available and are allowed to receive traffic in the VPC subnet endpoints
    • It selects a specific destination EC2 IP address and port.

In this example, it chooses a session running on port 81 of an EC2 instance with IP address 172.31.100.14 in a subnet in the AWS Availability Zone us-east-1a.

  1. From the stored mapping, the custom application logic looks up the accelerator and port that maps to the chosen EC2 instance and port.

In this example, the application returns the accelerator port “117” to the user.

  1. The user connects to Global Accelerator IP address and port 117. Note that the user can connect to port 117 on either of the two IP addresses (1.2.3.4 or 5.6.7.8) assigned to Global Accelerator.
  2. Global Accelerator translates the IP address/port to the destination IP address/port, 172.31.100.14:81.
  3. The user is routed to the private IP address/port of the actual game session on the EC2 server: 172.31.100.14:81.

Standard accelerators and custom routing accelerators

An accelerator in Global Accelerator always directs traffic to endpoints over the AWS global network to improve the availability and performance of internet applications. However, there are key differences between standard accelerators and custom routing accelerators. When you create an accelerator, you choose the type that is useful for your scenario.

Standard accelerator Custom routing accelerator
Routing Automatically picks the closest and healthy endpoint. Uses traffic dials, endpoint weights and client affinity to manage traffic. IP address and Port of the destination EC2 instance the user traffic should be routed to.
Endpoints Application Load Balancers (ALBs), Network Load Balancers (NLBs), EC2 instances, and Elastic IPs (EIPs). VPC subnets (traffic is routed only to EC2 instances in the subnets).
Protocol selection TCP OR UDP at Listener level. TCP AND/OR UDP at Endpoint group level.

Conclusion

In this blog post, we’ve described how to set up and use the new custom routing accelerators. Custom routing extends the performance benefits of AWS Global Accelerator to use cases where you want to route anywhere from a few to thousands of users to specific EC2 endpoint resources. We hope that you’ve found this post useful and we look forward to hearing how you use this feature!

About the authors

Jibril Touzi Jibril Touzi is a Sr. Edge Specialist Solutions Architect at AWS. Helping customers innovate using AWS Serverless and Edge services is what keeps him motivated. Jibril is a passionate photographer and enjoys spending time with family in outdoor activities when he is not working.
Vivek Sharma Vivek Sharma is Senior Solutions Architect for AWS Global and Strategic customers. He is passionate about helping developers build performant applications on AWS. He is an experienced technologist with a background in analytics, media workflows, performance engineering, security and content delivery networks.