Amazon CloudFront – Support for Dynamic Content

Post updated on May 11, 2020.

Introduction
Amazon CloudFront‘s network of edge locations (currently 30, with more in the works) gives you the ability to distribute static and streaming content to your users at high speed with low latency.

Today we are introducing a set of features that, taken together, allow you to use CloudFront to serve dynamic, personalized content more quickly.

What is Dynamic Personalized Content?
As you know, content on the web is identified by a URL, or Uniform Resource Locator such as https://media.amazonwebservices.com/blog/console_cw_est_charge_service_2.png . A URL like this always identifies a unique piece of content.

A URL can also contain a query string. This takes the form of a question mark  (“?”) and additional information that the server can use to personalize the request. Suppose that we had a server at www.example.com, and that can return information about a particular user by invoking a PHP script that accepts a user name as an argument, with URLs like http://www.example.com/userinfo.php?jeff or http://www.example.com/userinfo.php?tina.

Up until now, CloudFront did not use the query string as part of the key that it uses to identify the data that it stores in its edge locations.

We’re changing that today, and you can now use CloudFront to speed access to your dynamic data at our current low rates, making your applications faster and more responsive, regardless of where your users are located.

With this change (and the others that I’ll tell you about in a minute), Amazon CloudFront will become an even better component of your global applications. We’ve put together a long list of optimizations that will each increase the performance of your application on their own, but will work even better when you use them in conjunction with other AWS services such as Route 53, Amazon S3, and Amazon EC2.

Tell Me More
Ok, so here’s what we’ve done:

Persistent TCP Connections – Establishing a TCP connection takes some time because each new connection requires a three-way handshake between the server and the client. Amazon CloudFront makes use of persistent connections to each origin for dynamic content. This obviates the connection setup time that would otherwise slow down each request. Reusing these “long-haul” connections back to the server can eliminate hundreds of milliseconds of connection setup time. The connection from the client to the CloudFront edge location is also kept open whenever possible.

Support for Multiple Origins – You can now reference multiple origins (sources of content) from a single CloudFront distribution. This means that you could, for example, serve images from Amazon S3, dynamic content from EC2, and other content from third-party sites, all from a single domain name. Being able to serve your entire site from a single domain will simplify implementation, allow the use of more relative URLs within the application, and can even get you past some cross-site scripting limitations.

Support for Query Strings – CloudFront now uses the query string as part of its cache key. This optional feature gives you the ability to cache content at the edge that is specific to a particular user, city (e.g. weather or traffic), and so forth. You can enable query string support for your entire website or for selected portions, as needed.

Variable Time-To-Live (TTL) (Updated 5/20) – You can now set Minimum, Default, and Maximum TTL values in Amazon CloudFront distribution to achieve different caching results. If your Origin adds Cache Control headers such as max-age, s-maxage etc., then refer to this table to understand how CloudFront TTL and different caching directives interact together.

Large TCP Window – We increased the initial size of CloudFront’s TCP window to 10 back in February, but we didn’t say anything at the time. This enhancement allows more data to be “in flight” across the wire at a given time, without the usual waiting time as the window grows from the older value of 2.

API and Management Console Support – All of the features listed above are accessible from the CloudFront APIs and the CloudFront tab of the AWS Management Console. You can now use URL patterns to exercise fine-grained control over the caching and delivery rules for different parts of your site.

Of course, all of CloudFront’s existing static content delivery features will continue to work as expected. GET and HEAD requests, default root object, invalidation, private content, access logs, IAM integration, and delivery of objects compressed by the origin.

Working Together
Let’s take a look at the ways that various AWS services work together to make delivery of static and dynamic content as fast, reliable, and efficient and possible (click on the diagram at right for an even better illustration):

• From Application / Client to CloudFront – CloudFronts request routing technology ensures that each client is connected to the nearest edge location as determined by latency measurements that CloudFront continuously takes from internet users around the world. Route 53 may be optionally used as a DNS service to create a CNAME from your custom domain name to your CloudFront distribution. Persistent connections expedite data transfer.
• Within the CloudFront Edge Locations – Multiple levels of caching at each edge location speed access to the most frequently viewed content and reduce the need to go to your origin servers for cacheable content.
• From Edge Location to Origin – The nature of dynamic content requires repeated back and forth calls to the origin server. CloudFront edge locations collapse multiple concurrent requests for the same object into a single request. They also maintain persistent connections to the origins (with the large window size). Connections to other parts of AWS are made over high-quality networks that are monitored by Amazon for both availability and performance. This monitoring has the beneficial side effect of keeping error rates low and window sizes high.

• Path Pattern – A pattern (e.g. “*.jpg”) that identifies the content subject to this behavior.
• Origin Identifier -The identifier for the origin where CloudFront should forward user requests that match this path pattern.
• Query String – A flag to enable support for query string processing for URLs that match the path pattern.
• Trusted Signers – Information to enable other AWS accounts to create signed URLs for this URL path pattern.
• Protocol Policy – Either allow-all or https-only, also applied only to this path pattern.
• MinTTL – The minimum time-to-live for content subject to this behavior.

Tool Support
Andy from CloudBerry Lab sent me a note to let me know that they have added dynamic content support to the newest free version of the CloudBerry Explorer for Amazon S3.  In Andy’s words:

I’d like to let you know that CloudBerry Explorer is ready to support new CloudFront features by the time of release.  We have added the ability to manage multiple origins for a distribution, configure cache behavior for each origin based on URL path patterns and configure CloudFront to include query string parameters.

You can read more about this in their new blog post, How to configure CloudFront Dynamic Content with CloudBerry S3 Explorer .

Andy also sent some screen shots to show us how it works. The first step is to specify the Origins and CNAMEs associated with the distribution:

The next step is to specify the Path Patterns:

With the Origins and Path Patterns established, the final step is to configure the Path Patterns:

Update: Tej from Bucket Explorer wrote in to tell me that they are now supporting this feature:

Hi, I am one of the developer of Bucket Explorer. I am excited to announce that Bucket Explorer new version is supporting CloudFront Dynamic Content feature. Try Its 30 day trial version with full featured. Dynamic Content (Steps and Images).

And Here You Go
Together with CloudFront’s cost-effectiveness (no minimum commits or long-term contracts), these features add up to a content distribution system that is fast, powerful, and easy to use.

So, what do you think? What kinds of applications can you build with these powerful new features?

— Jeff;

PS – Read more about this new feature in Werner’s new post: Dynamic Content Support in Amazon CloudFront.

Modified 04/10/2023 – In an effort to ensure a great experience, expired links in this post have been updated or removed from the original post.