Scalability and familiarity with Amazon Keyspaces

Introduction

In this lesson, you create a Users service for your application. This service handles all user-specific functionality for your application, including registration and basic retrieval. For data storage, your service uses Amazon Keyspaces (for Apache Cassandra), a fully managed database that is compatible with Apache Cassandra.

This lesson teaches you how to use a fully managed Amazon Keyspaces table in an application. First, you learn why you would want to use Amazon Keyspaces. Then you work through the steps to create an Amazon Keyspaces table and use it in your application. At the end of this lesson, you should feel confident in your ability to use Amazon Keyspaces in your application.

Time to complete: 3045 minutes

Purpose-built Databases - Keyspaces (17:59)
Why use Amazon Keyspaces?

Amazon Keyspaces is a fully managed service for running Cassandra-compatible databases with AWS. Apache Cassandra is a popular option for high-scale applications that need top-tier performance.

Amazon Keyspaces is good for high-volume applications with straightforward access patterns. Many developers use Amazon Keyspaces for metadata use cases, such as a user-profile service, or for time-series data, such as storing data from an Internet of Things (IoT) device or handling messages in a messaging service. In this lesson, you use Amazon Keyspaces to handle user metadata because most of the access patterns are simple key-value access patterns.

You might choose Amazon Keyspaces if you have an existing application that is using Cassandra where you would like to reduce your operations burden by using a fully managed service. With Amazon Keyspaces, your database operations are managed by AWS, leaving your team free to focus on innovation. Amazon Keyspaces handles cluster scaling, instance failover, data backups, and software updates. If you want to see more about migrating an existing Cassandra workload to Amazon Keyspaces, see the lesson about migrating from self-managed Cassandra to fully managed Amazon Keyspaces.

You also might choose to build a greenfield application by using Amazon Keyspaces because of your familiarity with the API and data model. Amazon Keyspaces is compatible with the Apache Cassandra API, so your developers don’t need to learn a new API if they are already familiar with Cassandra. With Amazon Keyspaces, they can get the benefits of a familiar, open-source API while also using a fully managed database.

Lesson contents

In this lesson, you learn how to build a Users service by using Amazon Keyspaces as your data storage. This lesson has four parts.

  • 1. Create an AWS Cloud9 environment

    In this module, you create and prepare an AWS Cloud9 environment. AWS Cloud9 is a cloud-based integrated development environment (IDE). It gives you a fast, consistent development environment from which you can quickly build AWS applications.


    To get started, navigate to the AWS Cloud9 console. Choose Create environment to start the AWS Cloud9 environment creation wizard.

    Screenshot of AWS Cloud9
    (click to zoom)

    On the first page of the wizard, give your environment a name and a description. Then choose Next step.

    Give your environment a name and description
    (click to zoom)

    The next step allows you to configure environment settings, such as the instance type for your environment, the platform, and network settings.

    The default settings work for this lesson, so scroll to the bottom and choose Next step.

    Retain the default environment settings for this lesson
    (click to zoom)

    The last step shows your settings for review. Scroll to the bottom and choose Create environment.

    Choose "Create environment"
    (click to zoom)

    Your AWS Cloud9 environment should take a few minutes to provision. As it is being created, the following screen is displayed.

    The AWS Cloud9 environment as it is provisioning
    (click to zoom)

    After a few minutes, you should see your AWS Cloud9 environment. There are three areas of the AWS Cloud9 console to know, as illustrated in the following screenshot:

    • File explorer: On the left side of the IDE, the file explorer shows a list of the files in your directory.
    • File editor: In the upper right area of the IDE, the file editor is where you view and edit files that you’ve chosen in the file explorer.
    • Terminal: In the lower right area of the IDE, the terminal is where you run commands to execute code samples.
    The three main areas of the AWS Cloud9 console
    (click to zoom)

    In this lesson, you use Python to interact with your Amazon Keyspaces table. Run the following commands in your AWS Cloud9 terminal to download and unpack the module code.

    cd ~/environment
    curl -sL https://s3.amazonaws.com/aws-data-labs/keyspaces.tar | tar -xv

    Run the following command in your AWS Cloud9 terminal to view the contents of your directory.

    ls

    You should see seven files in your AWS Cloud9 terminal:

    • create_user.py
    • delete_user.py 
    • entities.py 
    • get_user.py
    • requirements.txt
    • test_connection.py
    • utils.py

    Run the following command in your terminal to install the dependencies for your application.

    sudo pip install -r requirements.txt

    Also, set your AWS Region in an environment file.

    echo "export AWS_REGION=us-east-1" >> env.sh && source env.sh

    This example uses us-east-1, but enter whichever AWS Region you want to use for this lesson.

    Use the env.sh file to store environment variables of resources and other parameters you need in this lesson. If you take a break during this lesson and start a new session in your AWS Cloud9 environment, be sure to reload your environment variables by executing the following command in your terminal.

    source env.sh


    In this module, you configured an AWS Cloud9 instance to use for development. In the next module, you create an Amazon Keyspaces table.

  • 2. Create an Amazon Keyspaces table

    In this module, you create an Amazon Keyspaces table. This table will be used as the primary database for the Users service in your application.


    To get started, navigate to the Amazon Keyspaces console. On the Amazon Keyspaces page, choose Create keyspace to create a new keyspace.

    Choose "Create keyspace" to create a new keyspace
    (click to zoom)

    In the keyspace creation wizard, give your keyspace the name users.

    Give your keyspace the name "users"
    (click to zoom)

    You may attach tags to your keyspace to help with access control or to help track billing. Then choose Create keyspace to create your keyspace.

    Choose "Create keyspace" to create your keyspace
    (click to zoom)

    At this point, you should see your keyspace in the Amazon Keyspaces console. Choose the keyspace to view more details about it.

    Choose the keyspace to view more details about it
    (click to zoom)

    Currently, your keyspace does not have any tables. Create the first table in your keyspace to hold users’ data in your application.

    Choose Create table to open the table-creation wizard.

    Choose "Create table" to open the table-creation wizard.
    (click to zoom)

    First, you need to choose a name for your table. Because your table will hold information about users, name the table users.

    Name the table "users"
    (click to zoom)

    Second, you need to declare the schema for your table. Add the following five columns to your table:

    • username
    • name
    • email
    • address
    • birthdate

    All columns should be of type ASCII except for birthdate, which should be of type timestamp.

    Your schema should look as follows in the Amazon Keyspaces table-creation wizard.

    Declare your table's schema in the table-creation wizard
    (click to zoom)

    Next, choose the partition key and any clustering columns for your table. The partition key is how you uniquely identify a specific item in your table.

    Choose username as the partition key because the username uniquely identifies each user.

    Choose "username" as the partition key
    (click to zoom)

    Finally, choose the billing mode and add any required tags. With Amazon Keyspaces on-demand capacity mode, you don't need to plan for the capacity required by your table. Amazon Keyspaces bills you directly for the reads and writes you consume. With provisioned capacity mode, you declare the amount of reads and writes you want to provision.

    Choose the billing mode and add any required tags
    (click to zoom)

    The table-creation wizard shows you the Apache Cassandra command to be executed to create your table.

    When you are ready, choose Create table to create your table.

    Choose "Create table" to create your table
    (click to zoom)

    The Amazon Keyspaces console shows your table being created. While the table is being created, it has a Status of Creating.

    When your table is ready to use, its Status is Active.

    While the table is being created, it has a "Status" of "Creating"
    (click to zoom)
    When your table is ready to use, its "Status" is "Active"
    (click to zoom)

    To connect to your Amazon Keyspaces table, create service-specific credentials. These credentials are one of the two ways you can authenticate to your Amazon Keyspaces table.

    Service-specific credentials are credentials tied to a specific AWS Identity and Access Management (IAM) user that are used to authenticate for a service.

    To create service-specific credentials, navigate to the IAM Users page of the AWS Management Console. Find the IAM user to whom you want to grant service-specific credentials and choose it.

    Find the IAM user to whom you want to grant service-specific credentials and choose it
    (click to zoom)

    On the IAM user's page, choose the Security credentials tab.

    On the IAM user's page, choose the "Security credentials" tab
    (click to zoom)

    Then navigate to the bottom of the page. In the section for Amazon Keyspaces, choose Generate credentials to create Amazon Keyspaces credentials for the IAM user.

    Choose "Generate credentials" to create Amazon Keyspaces credentials for the IAM user
    (click to zoom)

    A window is displayed with your service-specific credentials. Download these credentials and make sure you have them available because you need them later in this lesson.

    Download the credentials and make sure you have them available because you need them later in this lesson
    (click to zoom)

    After you download your service-specific credentials, save them to your env.sh file by entering the following commands in your AWS Cloud9 terminal.

    echo "export CASSANDRA_USERNAME=<username>" >> env.sh && source env.sh
    echo "export CASSANDRA_PASSWORD=<password>" >> env.sh && source env.sh

    Be sure to use the username and password from your credentials file.

    Next, download the Amazon digital certificate to your AWS Cloud9 environment.

    curl https://www.amazontrust.com/repository/AmazonRootCA1.pem -O

    There is a script named test_connection.py in the .zip file that you downloaded. This file ensures you can connect to your Amazon Keyspaces table. Open the file in your file editor to see its contents. You should see the following.

    from entities import User
    from utils import cluster
    
    session = cluster.connect()
    r = session.execute("SELECT * FROM users.users LIMIT 5")
    print(
        f"Connected to Amazon Keyspaces table. There are {len(r.current_rows)} rows in the results."
    )
    for row in r.current_rows:
        print(User(**row))

    The file imports a cluster from the utils.py module, and then uses the cluster to create a session and query the users table. It then prints out some information about the results.

    Let's take a look at the utils.py module to see how to create a cluster. Open the utils.py module in your AWS Cloud9 file editor. You should see the following contents.

    import os
    
    from cassandra.cluster import Cluster, ExecutionProfile, EXEC_PROFILE_DEFAULT
    from cassandra import ConsistencyLevel
    from cassandra.query import dict_factory
    from ssl import SSLContext, PROTOCOL_TLSv1, CERT_REQUIRED
    from cassandra.auth import PlainTextAuthProvider
    
    USERNAME = os.environ["CASSANDRA_USERNAME"]
    PASSWORD = os.environ["CASSANDRA_PASSWORD"]
    REGION = os.environ["AWS_REGION"]
    
    ssl_context = SSLContext(PROTOCOL_TLSv1)
    ssl_context.load_verify_locations("./AmazonRootCA1.pem")
    ssl_context.verify_mode = CERT_REQUIRED
    auth_provider = PlainTextAuthProvider(username=USERNAME, password=PASSWORD)
    profile = ExecutionProfile(
        consistency_level=ConsistencyLevel.LOCAL_QUORUM, row_factory=dict_factory
    )
    cluster = Cluster(
        [f"cassandra.{REGION}.amazonaws.com"],
        ssl_context=ssl_context,
        auth_provider=auth_provider,
        port=9142,
        execution_profiles={EXEC_PROFILE_DEFAULT: profile},
    )

    This module uses the standard Cassandra driver for Python to connect to Amazon Keyspaces and run a query. Much of it is similar to connecting to an Apache Cassandra database cluster, with three main differences:

    1. You create and use an SSLContext object that uses the Amazon digital certificate when creating a connection.
    2. You use the username and password from the service-based credentials to create a PlainTextAuthProvider that is used to authenticate to the Amazon Keyspaces table.
    3. You update some settings on the default execution profile to use a consistency level of LOCAL_QUORUM, which is supported by Amazon Keyspaces.

    It is helpful to put this connection creation logic in a utility module as we’ve done here so that you don't need to go through the steps to create an Amazon Keyspaces connection in each module where you connect to your database.


    In this module, you created a keyspace and table in Amazon Keyspaces. This table is fully managed and compatible with Apache Cassandra. You then created some service-specific credentials for authenticating with Amazon Keyspaces. Finally, you saw how to connect to Amazon Keyspaces and ran a script to test your connection.

    In the next module, you see how to implement some of the data access logic around user creation with Amazon Keyspaces.

  • 3. Insert, read, and delete data with Amazon Keyspaces

    In this module, you learn how to insert, read, and delete data with Amazon Keyspaces. This is similar to the data manipulation you do in your Users service.


    First, insert your first user into your Amazon Keyspaces table. In the files you downloaded to your AWS Cloud9 environment, there is a file called create_user.py. Open that file in your file editor. You should see the following contents.

    from entities import User
    from utils import cluster
    
    session = cluster.connect()
    
    
    def create_user(user):
        print(f"Creating {user.username}...")
        command = f"INSERT INTO users.users (username, name, email, address, birthdate) VALUES ('{user.username}', '{user.name}', '{user.email}', '{user.address}', '{user.birthdate}') IF NOT EXISTS"
        r = session.execute(command)
        results = r.one()
        applied = results["[applied]"]
        if not applied:
            raise Exception(f"User {user.username} already exists")
        print(f"User {user.username} created successfully!")
        return user
    
    
    user = User(
        username="michelle72",
        name="Thomas Davis",
        email="jake74@hotmail.com",
        address="136 Ryan Knolls, East Tiffany, AL 29438",
        birthdate="1986-07-14",
    )
    
    create_user(user)

    Let's walk through this module. After importing the libraries, there is a function called create_user. This is similar to a function that you would have in your Users service. It takes in a user object and saves the user to your Amazon Keyspaces table.

    As part of the INSERT query, it inserts the user if a user with that username does not already exist. This ensures a chosen username is unique to a specific user. After receiving the results of the query, the function throws an error if a user with that username already exists. Otherwise, it returns the saved user.

    After defining the create_user function, the module then instantiates a user object in your application and attempts to save it by calling the create_user function.

    Try executing the function to save the first user. In your AWS Cloud9 terminal, execute the following command to execute the script.

    python create_user.py

    You should see the following output in your terminal.

    $ python create_user.py
    Creating michelle72...
    User michelle72 created successfully!

    Great! The user was created.

    To test the logic around the unique username requirement, execute the script again to try to create the user again.

    python create_user.py

    This time, you should see an error in your terminal.

    $ python create_user.py
    Creating michelle72...
    Traceback (most recent call last):
    File "create_user.py", line 27, in <module>
    create_user(user)
    File "create_user.py", line 14, in create_user
    raise Exception(f"User {user.username} already exists")
    Exception: User michelle72 already exists

    You can see the function threw an error because the user already exists in the database.

    Now, retrieve the user from the database. Open the get_user.py file in the file editor. You should see the following contents.

    from entities import User
    from utils import cluster
    
    session = cluster.connect()
    
    
    def get_user(username):
        print(f"Retrieving {username}...")
        command = f"SELECT * FROM users.users WHERE username = '{username}'"
        r = session.execute(command)
        results = r.one()
        if not results:
            raise Exception(f"User {username} does not exist")
        user = User(**results)
        print(user)
        return user
    
    
    user = get_user("michelle72")

    The contents are similar to the create_user.py file. There is a function, get_user, that retrieves a user by a given username from your Amazon Keyspaces table. This is similar to the function that will be used in your Users service. At the end of the file, there is a statement to call the get_user function with the username of the user that you created.

    Execute the file with the following command.

    python get_user.py

    You should see the following output.

    $ python get_user.py
    Retrieving michelle72...
    User<michelle72 -- Thomas Davis>

    Success! You retrieved the user from Amazon Keyspaces and constructed a user object in your application.

    Finally, delete your record from the database. Open the delete_user.py file in your file editor. You should see the following contents.

    from entities import User
    from utils import cluster
    
    session = cluster.connect()
    
    
    def delete_user(username):
        print(f"Deleting {username}...")
        command = f"DELETE FROM users.users WHERE username = '{username}'"
        r = session.execute(command)
        results = r.one()
        print(f"User {username} deleted.")
    
    
    delete_user("michelle72")

    This file is similar to the previous two. There is a delete_user function that is similar to what you use in your application code. It takes a username and deletes the user from your Amazon Keyspaces table. Additionally, there is a statement at the end of the file to execute the function and delete your user.

    Execute the file with the following command.

    python delete_user.py

    You should see the following output.

    $ python delete_user.py
    Deleting michelle72...
    User michelle72 deleted.

    Success! You deleted your user from Amazon Keyspaces.

    Try executing the get_user.py file again. You should now get an error because your user no longer exists in your Amazon Keyspaces table.

    $ python get_user.py
    Retrieving michelle72...
    Traceback (most recent call last):
    File "get_user.py", line 19, in <module>
    user = get_user("michelle72")
    File "get_user.py", line 13, in get_user
    raise Exception(f"User {username} does not exist")
    Exception: User michelle72 does not exist


    In this module, you learned how to insert, read, and delete data from Amazon Keyspaces. In doing so, you learned about some example functions that you would have in the application code for your Users service.

    In the next module, you clean up the resources you have created in this lesson.

  • 4. Clean up the resources you created

    In this lesson, you created an Amazon Keyspaces table to be used as the primary database for a Users service in your application. Amazon Keyspaces is a great fit for metadata services such as user profiles. Amazon Keyspaces provides fast, reliable performance and scales up with your application, and it is compatible with the open-source Apache Cassandra API, providing a consistent experience for developers that have used Cassandra in the past.

    In this module, you clean up the resources you created in this lesson to avoid incurring additional charges.


    First, delete your Amazon Keyspaces table and keyspace. To do so, navigate to the Amazon Keyspaces console. Choose the keyspace you created, and then choose Delete.

    Delete your Amazon Keyspaces table and keyspace
    (click to zoom)

    On the deletion confirmation page, type Delete in the box, and then choose Delete keyspace to delete your keyspace.

    Type &quot;Delete&quot; in the box, and then choose &quot;Delete keyspace&quot; to delete your keyspace
    (click to zoom)

    The Amazon Keyspaces page shows that your keyspace is being deleted. Note that your table is under the keyspace and is deleted along with the keyspace.

    Your table is deleted along with the keyspace
    (click to zoom)

    Additionally, you need to delete your AWS Cloud9 development environment. To do so, navigate to the AWS Cloud9 console. Choose the environment you created for this lesson, and then choose Delete.

    Delete your AWS Cloud9 development environment
    (click to zoom)

    In this module, you learned how to clean up the Amazon Keyspaces resources and the AWS Cloud9 environment that you created in this lesson.

In this lesson, you learned how to create and use an Amazon Keyspaces table in your application. First, you created the Amazon Keyspaces table and specified the columns in your table. Then you learned how to authenticate and connect to your Amazon Keyspaces table. Finally, you learned how to create, read, and delete data in your Amazon Keyspaces table. You can use these patterns when building applications with Amazon Keyspaces.