Application Performance - High availability architectures

AWS Edge services are important components for building web applications with high availability. In addition to their native high availability thanks to their distributed nature, AWS Edge services act as entry point to web applications, and can route requests to available partitions in your origin infrastructure (e.g availability zone or region).

A high availability web application can be architected in a variety of ways, based on your requirements in terms of availability SLO, cost, and complexity. At minimum, you need to make the following technical decisions based on your business requirements:

CloudFront can help you mitigate the impact of transient 5xx errors that can occasionally impair a small number of user requests. Often this is due to an overwhelmed origin by sudden traffic spikes, or transient network issues. You can mitigate transient 5xx errors with CloudFront in different ways.

Retry. CloudFront can be configured to retry TCP connection up to three times with a configurable connection timeout when a connection can't be established with the origin. CloudFront also retry idempotent GET/HEAD according to the configured number of retries.

Respond with stale content. If the retries fail with the origin, CloudFront by default serves a stale version of the object if available in cache instead of returning an error response. This behavior can be disabled using Cache-Control: stale-if-error=0 directive.

Origin Failover to secondary origin. You can configure CloudFront to fail to a secondary origin for the specific failed request using Origin Failover functionality. Origin Failover only applies to idempotent GET/HEAD requests. Note that if you use Lambda@Edge on origin events, CloudFront executes the function on failover as well. In this scenario, you can infer in the Lambda@Edge function whether the request was for the primary or the secondary origin to differentiate its logic. To do that, configured the same Origin Custom Header in both origins, but with different values that Lambda@Edge can read.

Graceful failover. If Origin Failover to another origin is not desirable (e.g. maintain another origin infrastructure), consider failing over to a static file hosted in a different location (e.g. S3 bucket) using Custom Error Page. By default, the same page is used for all requests resulting in errors, however you build a more dynamic response, by following the third pattern in this blog.

You can use Route 53 Failover routing policy with health checks on the origin domain name that’s configured as the origin in CloudFront. When the primary origin becomes unhealthy, Route 53 detects it, and then starts resolving the origin domain name with the IP address of the secondary origin. CloudFront honors the origin DNS TTL, which means that traffic will start flowing to the secondary origin within the DNS TTL. The most optimal configuration (Fast Check activated, a failover threshold of 1, and 60 second DNS TTL) means that the failover will take 70 seconds at minimum to occur. When it does, all of the traffic is switched to the secondary origin, since it’s a stateful failover. Note that this design can be further extended with Route 53 Application Recovery Control for more sophisticated application failover across multiple AWS Regions, Availability Zones, and on-premises. This approach can be combined with CloudFront Origin Failover functionality for GET/HEAD requests to reduce errors while Route 53 fails over to the secondary origin. This pattern is described in this blog.

When the different origins don't share the same domain name, such as for S3 based origins, Route 53 can't be used in the proposed way above. In this scenario, you need a Lambda@Edge function configured on origin request event to query Route 53 for which origin to select, then reroute the request to the selected origin, by changing the origin domain name and with the Host header. To learn more about this implementation, read the following Contentful Case Study.

Application that are using Global Accelerator can benefit from its native failover functionalities. Using Global Accelerator, your application can be deployed in a single AWS Region across multiple Availability Zones or across multiple AWS Regions. Global Accelerator monitors the health of your application endpoints using health checks, and routes traffic away from unhealthy endpoints within tens of seconds.

CloudFront can be used with Route 53 latency-based policy to route user requests to nearest AWS region where you have replicated origin in an Active-Active multi-region architecture. To do this, configure the origin domain name in CloudFront with latency based policy in Route 53. When CloudFront resolves the origin domain name to connect and send the request to the origin, Route 53 returns the nearest origin IP based on latency. Note that the location from which CloudFront resolves the origin domain name depends on the distribution configuration and the traffic type. In general, the domain name is resolved in the edge location for dynamic requests, and in Regional Edge Caches for cacheable content. This blog guides you through an active-active multi-region deployment for API Gateway.

When the different origins don't share the same domain name, such as for S3 based origins, Route 53 can't be used in the above proposed way. In this scenario, you need a Lambda@Edge configured on origin request event to route to the nearest origin. To learn more about this implementation, read the followings blogs:

In certain scenarios, request routing requires application level logic. When the logic is simple such as routing to different origins based on path (e.g. route /api1 to origin 1 and /api2 to origin 2), you can simply use the native cache behavior capability of CloudFront.

If the routing logic is more sophisticated (e.g. based on user subscription tier), you can use CloudFront's edge functions. With CloudFront Functions, you can fetch your route definitions from KeyValueStore, and change the routing with the updateRequestOrigin() method. If the routing logic requires access to external sources, use Lambda@Edge on origin request event, and update the origin attribute of the event object in the function handler.