Caching for high-volume workloads with Amazon ElastiCache

To get started, navigate to the AWS Cloud9 console. Choose Create environment to start the AWS Cloud9 environment creation wizard.

On the first page of the wizard, give your environment a name and a description. Then choose Next step.

The next step allows you to configure environment settings, such as the instance type for your environment, the platform, and network settings.

The default settings work for this lesson, so scroll to the bottom and choose Next step.

The last step shows your settings for review. Scroll to the bottom and choose Create environment.

Your AWS Cloud9 environment should take a few minutes to provision. As it is being created, the following screen is displayed.

After a few minutes, you should see your AWS Cloud9 environment. There are three areas of the AWS Cloud9 console to know, as illustrated in the following screenshot:

• File explorer: On the left side of the IDE, the file explorer shows a list of the files in your directory.
• File editor: In the upper right area of the IDE, the file editor is where you view and edit files that you’ve chosen in the file explorer.
• Terminal: In the lower right area of the IDE, the terminal is where you run commands to execute code samples.
  

First, navigate to the ElastiCache console. Choose Create to begin the instance-creation wizard.

In the Cluster engine settings, choose Redis.

In the Redis settings section, give your Redis instance a name. Then change the Node type to cache.t2.micro and the Number of replicas to 0. In a production setting, you might choose to use replicas and a larger Node type.

In the Advanced Redis settings, you can leave the first few settings with their default options.

Under the Security section, take note of the security group that is attached to your Redis instance. You will edit this after your cluster is created to allow access to Redis from your AWS Cloud9 instance.

The default security group is chosen and works fine for this lesson.

The rest of the default settings are fine. Scroll to the bottom and choose Create to create your Redis instance.

ElastiCache begins creating your Redis instance. While your instance is being created, it shows a Status of creating in the ElastiCache console.

When your ElastiCache instance is ready, its Status is available.

Next, you need to allow inbound access to the security group for your Redis instance from your AWS Cloud9 environment. This allows you to connect to your Redis instance.

Navigate to the Security Groups page of the Amazon EC2 console. You should see the default security group you used for your ElastiCache instance as well as a security group for your AWS Cloud9 instance.

Choose the default security group to see more details.

After you choose the default security group, you should see details about your group as well as the inbound rules for your security group. Choose Edit inbound rules to edit the rules.

Add an inbound rule to allow traffic to your Redis instance from your AWS Cloud9 instance. The Type of your rule should be Custom TCP with a Port range of 6379. For the Source, choose the security group used for your AWS Cloud9 instance.

Your screen should look as follows.

Choose Save rules to save your new inbound security group rules.

Finally, test the connection to your Redis instance from your AWS Cloud9 environment. In the ElastiCache console, find your instance and expand the details. Find the value for Primary endpoint and copy the value.

Set the value of the endpoint in your AWS Cloud9 environment by running the following command in your terminal.

export REDIS_HOSTNAME=<yourPrimaryEndpoint>

Be sure to replace <yourPrimaryEndpoint> with the primary endpoint you copied from the ElastiCache console.

There is a script called test_connection.py to test that you can connect to Redis. The contents of the script are as follows.

import os

import redis

HOST = os.environ["REDIS_HOSTNAME"].replace(":6379", "")

r = redis.Redis(host=HOST)

r.ping()

print("Connected to Redis!")

The script uses Python's redis library to connect to Redis. It uses the hostname from the REDIS_HOSTNAME environment you set in your terminal to initialize a connection to Redis. It then runs a PING command against the Redis server. If the connection was successful, you should see console output indicating that you could connect. If you see an exception, review the preceding steps to ensure you set your environment variable correctly and allowed inbound access from your AWS Cloud9 environment.

from dynamodb import fetch_restaurant_summary_from_database
from cache import fetch_restaurant_summary_from_cache, store_restaurant_summary_in_cache


def fetch_restaurant_summary(restaurant_name):
    restaurant = fetch_restaurant_summary_from_cache(restaurant_name)
    if restaurant:
        print("Using cached result!")
        return restaurant

    restaurant = fetch_restaurant_summary_from_database(restaurant_name)
    store_restaurant_summary_in_cache(restaurant)

    print("Using uncached result!")

    return restaurant


restaurant = fetch_restaurant_summary("The Vineyard")

print(restaurant)
for review in restaurant.reviews:
    print(review)

This file includes a function, fetch_restaurant_summary, that is similar to a function you would have in your application. The function takes a restaurant name and performs the following actions:

1. It checks the Redis cache for the restaurant summary. If the data exists in the cache, it is returned to the user.
2. If the data does not exist in the cache, the function retrieves the restaurant summary from DynamoDB. This would use code similar to that used to handle the Fetch Restaurant Summary access pattern in the DynamoDB lesson.
3. After receiving the results from DynamoDB, the function caches the results in Redis for future requests.
4. The summary is returned to the user.

There is a file named cache.py that contains the code for interacting with the Redis cache. Open the file in your file editor. The contents look as follows.

from entities import Restaurant, Review

import json
import os

import redis

HOST = os.environ["REDIS_HOSTNAME"].replace(":6379", "")

r = redis.Redis(host=HOST)


class ObjectEncoder(json.JSONEncoder):
    def default(self, o):
        return o.__dict__


def store_restaurant_summary_in_cache(restaurant):
    key = restaurant.name
    r.set(key, json.dumps(restaurant, cls=ObjectEncoder), ex=900)

    return True


def fetch_restaurant_summary_from_cache(restaurant_name):
    response = r.get(restaurant_name)
    if response:
        data = json.loads(response)
        restaurant = Restaurant(data)
        restaurant.reviews = [Review(review) for review in data["reviews"]]
        return restaurant
    return None

There are two functions here. The first function is for storing a restaurant summary in Redis. It uses the restaurant name as the key name in Redis and turns the restaurant summary into a JSON string. Then it uses the Redis SET command to set the value of a string key in Redis. When doing so, it uses a Time to Live (TTL) value of 900 so that the value expires in 15 minutes. This helps to ensure your data doesn't get too stale.

The second function is responsible for retrieving a restaurant summary from Redis. It takes a restaurant name and performs the Redis GET command to retrieve the value. It then parses the data, reconstitutes it into objects in your application, and returns the result.

You can test this cache usage by running the following command in your terminal.

python fetch_restaurant_summary.py

The first time you run the command, you should see the following output.

$ python fetch_restaurant_summary.py
Using uncached result!
Restaurant<The Vineyard -- Fine Dining>
Review<The Vineyard -- markmartin (2020-05-24T11:59:44)>
Review<The Vineyard -- kgraham (2020-05-14T15:01:52)>
Review<The Vineyard -- ewilliams (2020-05-13T02:36:30)>
Review<The Vineyard -- hannah21 (2020-05-04T03:44:26)>
Review<The Vineyard -- john97 (2020-04-27T20:45:52)>

Notice that the script indicates that it used an uncached result before printing out the restaurant and reviews.

Run the script again to test your caching behavior.

python fetch_restaurant_summary.py

This time you should see the following output.

$ python fetch_restaurant_summary.py
Using cached result!
Restaurant<The Vineyard -- Fine Dining>
Review<The Vineyard -- markmartin (2020-05-24T11:59:44)>
Review<The Vineyard -- kgraham (2020-05-14T15:01:52)>
Review<The Vineyard -- ewilliams (2020-05-13T02:36:30)>
Review<The Vineyard -- hannah21 (2020-05-04T03:44:26)>
Review<The Vineyard -- john97 (2020-04-27T20:45:52)>

You can see that the script indicates it's using a cached result this time. The output is still the same as the initial fetch from DynamoDB though. Based on the TTL set on the key (expiration value of 900 seconds), for the next 15 minutes these results stay in your Redis instance to serve cached results quickly.

First, delete your ElastiCache instance. Navigate to the ElastiCache console. Find the ElastiCache instance you created for this module and select the check box. Then choose Delete to delete your instance.

In the confirmation window, decline the option to create a final backup for your Redis instance, and then choose Delete.

Additionally, you need to delete your AWS Cloud9 development environment.

To do so, navigate to the AWS Cloud9 console. Choose the environment you created for this lesson, and then choose Delete.