AWS Database Blog

Manage AWS ElastiCache for Redis access with Role-Based Access Control, AWS Secrets Manager, and IAM

October 2022: This post was reviewed and updated with a new architecture diagram and code updates to factor the change from CDK 1.x to CDK 2.x.

Amazon ElastiCache for Redis is an AWS managed, Redis-compliant service that provides a high-performance, scalable, and distributed key-value data store that you can use as a database, cache, message broker, or queue. Redis is a popular choice for caching, session management, gaming, leaderboards, real-time analytics, geospatial, ride-hailing, chat and messaging, media streaming, and pub/sub apps.

You can authenticate in ElastiCache for Redis in one of two ways: via an authentication token or with a username and password via Role-Based Access Control (RBAC) for ElastiCache for Redis 6 and later.

Although authentication with a token allows administrators to restrict reads and writes to ElastiCache replication groups via a password, RBAC enhances this by introducing the concept of ElastiCache users and groups. Additionally, with RBAC in Amazon ElastiCache for Redis 6, administrators can specify access strings for each ElastiCache user—further defining which commands and keys they can access.

When configured for RBAC, ElastiCache for Redis replication groups authenticate ElastiCache RBAC users based on the credentials provided when connections are established. Authorization to Redis commands and keys are defined by the access strings (in Redis ACL syntax) for each ElastiCache RBAC user.

ElastiCache RBAC users and ACLs, however, aren’t linked to AWS Identity Access Management (IAM) roles, groups, or users. The dissociation between IAM and Redis RBAC means that there is no out-of-the-box way to grant IAM entities (roles, users, or groups) read and write access to Redis.

In this post, we present a solution that allows you to associate IAM entities with ElastiCache RBAC users and ACLs. The overall solution demonstrates how ElastiCache RBAC users can effectively be associated with IAM through the use of AWS Secrets Manager as a proxy for granting access to ElastiCache RBAC user credentials.

Solution overview

To demonstrate this solution, we implement the following high-level steps:

  • Define a set of ElastiCache RBAC users; each with credentials and ACL access strings. This defines the commands and keys that a user has access to.
  • Grant IAM entities access to ElastiCache RBAC user credentials stored in Secrets Manager through secret policies and IAM policies.
  • Configure users, applications, and services with roles or users that can access ElastiCache RBAC user credentials from Secrets Manager so they can connect to ElastiCache Redis by assuming an ElastiCache RBAC user. This also defines which commands and keys they have access to.

Store Redis RBAC passwords in Secrets Manager

You can create ElastiCache RBAC users via the AWS Command Line Interface (AWS CLI), AWS API, or AWS CloudFormation. When doing so, they’re specified with a plaintext password and a username. These credentials must then be shared with the actors who access the Redis replication group via ElastiCache RBAC users (human users or applications).

The solution we present uses Secrets Manager to generate a password that is used when the ElastiCache RBAC user is created, meaning that no plaintext passwords are exposed and must be retrieved through Secrets Manager.

Manage access to ElastiCache RBAC user passwords in Secrets Manager with IAM

You can restrict access to the credentials stored in Secrets Manager to specific IAM entities by defining a secret resource policy in addition to IAM policies. IAM entities can then retrieve the credentials by making the appropriate AWS API or AWS CLI call. See the following code:

{
  "Version" : "2012-10-17",
  "Statement" : [ {
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : "arn:aws:iam::1234567890123:role/producer"
    },
    "Action" : [ "secretsmanager:DescribeSecret", "secretsmanager:GetSecretValue" ],
    "Resource" : "arn:aws:secretsmanager:us-west-2:1234567890123:secret:producerRBACsecret "
  } ]
}

Manage access to ElastiCache for Redis with ElastiCache RBAC, Secrets Manager, and IAM

In essence, we’re creating a mapping between IAM roles and ElastiCache RBAC users by defining which IAM roles, groups, and users can retrieve credentials from Secrets Manager. The following diagram demonstrates the flow of the solution.

First, an actor with an IAM role that has permissions to the secret (named Producer Credentials) reads it from Secrets Manager (Steps 1 and 2). Next, the actor establishes a connection with the credentials to an ElastiCache replication group (3). After the user is authenticated (4), they can perform commands and access keys (5)— the commands and keys that can be accessed are defined by the ElastiCache RBAC user’s access string.

Implementation in AWS Cloud Development Kit

We present the solution to you in the AWS Cloud Development Kit (AWS CDK), which is a software development framework that defines infrastructure through object-oriented programming languages—in our case, Typescript. You can clone the code from the GitHub repo.

The following is deployed:

  • One VPC with isolated subnets and one Secrets Manager VPC endpoint
  • One security group with an ingress rule that allows all traffic in via port 6379
  • Three ElastiCache RBAC users: default, consumer, producer
  • Three secrets: default, producer, consumer
  • One ElastiCache RBAC user group
  • One ElastiCache subnet group
  • One ElastiCache replication group
  • One AWS Key Management Service (AWS KMS) customer master key (CMK) to encrypt the three secrets
  • One KMS CMK key to encrypt the ElastiCache replication group
  • Three IAM roles: consumer, producer, outsider
  • One AWS Lambda layer that contains the redis-py Python module
  • Three Lambda functions: producerFn, consumerFn, outsiderFn

The following diagram illustrates this architecture.

A VPC is created to host the ElastiCache replication group and the Lambda functions. The code snippet defines the VPC with an isolated subnet, which in AWS CDK terms is a private subnet with no routing to the internet. For resources in the isolated subnet to access Secrets Manager, a Secrets Manager VPC interface endpoint is added. See the following code:

const vpc = new ec2.Vpc(this, "Vpc", {
      subnetConfiguration: [
        {
          cidrMask: 24,
          name: 'Isolated',
          subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
        }
      ]
    });

    const secretsManagerEndpoint = vpc.addInterfaceEndpoint('SecretsManagerEndpoint', {
      service: ec2.InterfaceVpcEndpointAwsService.SECRETS_MANAGER,
      subnets: {
        subnetType: ec2.SubnetType.PRIVATE_ISOLATED
      }
    });

secretsManagerEndpoint.connections.allowDefaultPortFromAnyIpv4();

To modularize the design of the solution, a RedisRbacUser class is also created. This class is composed of two AWS CDK resources: a Secrets Manager secret and an ElastiCache CfnUser; these resources are explicitly grouped together because the secret stores the CfnUser password, and as we show later, read and decrypt permissions to the secret are granted to an IAM user. See the following code:

export class RedisRbacUser extends cdk.Construct {
  ...

  constructor(scope: cdk.Construct, id: string, props: RedisRbacUserProps) {
    super(scope, id);

    ...

    this.rbacUserSecret = new secretsmanager.Secret(this, 'secret', {
      generateSecretString: {
        secretStringTemplate: JSON.stringify({ username: props.redisUserName }),
        generateStringKey: 'password',
        excludeCharacters: '@%*()_+=`~{}|[]\\:";\'?,./'
      },
    });

    const user = new elasticache.CfnUser(this, 'redisuser', {
      engine: 'redis', 
      userName: props.redisUserName,
      accessString: props.accessString? props.accessString : "off +get ~keys*", 
      userId: props.redisUserId,
      passwords: [this.rbacUserSecret.secretValueFromJson('password').unsafeUnwrap()]
    })

    ...

  }

}

An IAM role is granted the ability to read the RedisRbacUser’s secret. This association means that the IAM role can decrypt the credentials and use them to establish a connection with Redis as the producerRbacUser:

const producerRole = new iam.Role(this, producerName+'Role', {
      ...
    });

producerRbacUser.grantSecretRead(producerRole)

The function grantSecretRead in the RedisRbacUser class modifies the role that is passed into it to allow it to perform actions secretsmanager:GetSecretValue and secretsmanager:DescribeSecret. The same function also modifies the secret by adding a resource policy that allows the same actions and adds the role to the principal list. This prevents unlisted principals from attempting to access the secret after the stack is deployed. See the following code:

public grantReadSecret(principal: iam.IPrincipal){
    if (this.secretResourcePolicyStatement == null) {
      this.secretResourcePolicyStatement = new iam.PolicyStatement({
        effect: iam.Effect.ALLOW,
        actions: ['secretsmanager:DescribeSecret', 'secretsmanager:GetSecretValue'],
        resources: [this.rbacUserSecret.secretArn],
        principals: [principal]
      })

      this.rbacUserSecret.addToResourcePolicy(this.secretResourcePolicyStatement)

    } else {
      this.secretResourcePolicyStatement.addPrincipals(principal)
    }

    this.rbacUserSecret.grantRead(principal)
  }

A Lambda function uses the IAM role created previously to decrypt the credentials stored in the secret and access the ElastiCache for Redis replication group. The ElastiCache primary endpoint address and port as well as the secret ARN are provided via environment variables. See the following code:

const producerLambda = new lambda.Function(this, producerName+'Fn', {
      ...
      role: producerRole,
      ...
      environment: {
        redis_endpoint: ecClusterReplicationGroup.attrPrimaryEndPointAddress,
        redis_port: ecClusterReplicationGroup.attrPrimaryEndPointPort,
        secret_arn: producerRbacUser.getSecret().secretArn,
      }
    });

Deploy the solution

The infrastructure for this solution is implemented in AWS CDK in Typescript and can be cloned from the GitHub repository.

For instructions on setting up your environment for AWS CDK, see Prerequisites.

To deploy the solution, first install the node dependencies by navigating to the root of the project and running the following command in the terminal:

$ npm install

Next, build the Lambda .zip files that are used in the Lambda functions. To do so, enter the following command in your terminal:

$ npm run-script zip

To deploy the solution to your account, run the following command from the root of the project:

$ cdk deploy

The command attempts to deploy the solution in the default AWS profile defined in either your ~/.aws/config file or your ~/.aws/credentials file. You can also define a profile by specifying the --profile profile_name at the end of the command.

Test the solution

Three Lambda functions are deployed as a part of the stack:

  • Producer – Decrypts the producer credentials from Secrets Manager and establishes a connection to Redis with these credentials. After it’s authenticated as the producer RBAC user, the function attempts to set a key (time) with a string representation of the current time. If the function attempts to perform any other commands, it fails because the producer RBAC user only allows it to perform SET operations.
  • Consumer – Decrypts the consumer credentials from Secrets Manager and establishes a connection to Redis with these credentials. After it’s authenticated as the consumer RBAC user, the function attempts to get the value of the time key that was set by the producer. The function fails if it attempts to perform other Redis commands because the access string for the consumer RBAC user only allows it to perform GET operations.
  • Outside – Attempts to decrypt the producer credentials from Secrets Manager and fails because the function’s role doesn’t have permission to decrypt the producer credentials.

Create a test trigger for each function

To create a test event to test each function, complete the following steps:

  1. On the Lambda console, navigate to the function and choose Test.
  2. Select Create new test event.
  3. For Event template, choose test.
  4. Use the default JSON object in the body—the test functions don’t read the event contents.
  5. Trigger each test by choosing Test.

Producer function writes to Redis

The producer function demonstrates how you can use an IAM role attached to a Lambda function to retrieve an ElastiCache RBAC user’s credentials from Secrets Manager, and then use these credentials to establish a connection to Redis and perform a write operation.

The producer function writes a key time with a value of the current time.

The producer function can write to Redis because its IAM role allows it to get and decrypt the Producer credentials in Secrets Manager, and the Producer ElastiCache RBAC user’s access string was defined to allow SET commands to be performed.

The producer function can’t perform GET commands because the same access string doesn’t allow GET commands to be performed. See the following code:

const producerRbacUser = new RedisRbacUser(this, producerName+'RBAC', {
      ...
      accessString: 'on ~* -@all +SET'
    });

Consumer function can read but can’t write to Redis

This function demonstrates the use case in which you allow a specific IAM role to access ElastiCache RBAC credentials from Secrets Manager and establish a connection with Redis, but the actions it can perform are restricted by an access string setting.

The consumer function attempts to write a key time with a value of the current time, and subsequently attempts to read back the key time.

The consumer function can’t write to Redis, but it can read from it. Even though the function has an IAM role that permits it to get and decrypt the Consumer credentials in Secrets Manager, the Consumer ElastiCache RBAC user was created with a Redis ACL access string value that only allows the GET command. See the following code:

const consumerRbacUser = new RedisRbacUser(this, consumerName+'RBAC', {
      ...
      accessString: 'on ~* -@all +GET'
    });

Outsider function can’t read or write to Redis

The outsider function demonstrates the use case in which you specify an IAM role that can’t access Redis because it can’t decrypt credentials stored in Secrets Manager.

The outsider Lambda function attempts to decrypt the Producer credentials from Secrets Manager, then read and write to the Redis cluster.

An exception is raised that indicates that it’s not permitted to access the Producer secret. The IAM role attached to it doesn’t have the permissions to decrypt the Producer secret, and the secret it’s trying to decrypt has a resource policy that doesn’t list the role in the principals list attribute.

Cost of running the solution

The solution to associate an IAM entity with an ElastiCache RBAC user requires deploying a sample ElastiCache cluster, storing secrets in Secrets Manager, and defining an ElastiCache RBAC user and user group. To run this solution in us-east-1, you can expect the following costs. Please note that costs vary by region.

  • Secrets Manager
    • $0.40 per secret per month, prorated for secrets stored less than a month
    • $0.05 per 10,000 API calls
    • Assuming each of the three secrets are called 10 times for testing purposes in one day, the total cost is (3 * $0.40 / 30) + (3 * 10 / 1000) * $0.05 = $0.04015
  • ElastiCache
    • cache.m6g.large node $0.077 per hour
    • Assuming that the node used for one day, the total cost is $1.848
  • Lambda function
    • $0.0000000021 per millisecond of runtime
    • Assuming that each function is called 10 times for testing purposes in one day and that the average runtime is 400 milliseconds, the total cost is 3 * 400 * $0.000000021 = $0.00000252
  • AWS KMS
    • $1 per month, per key, prorated to the hour
    • $0.03 per 10000 API calls
    • Assuming that the solution is torn down after 24 hours, the total cost for two keys is 2 * 1 / 31 = $0.06

The total cost of the solution, for 24 hours, assuming that each of the three Lambda functions are called 10 times, is $1.95.

Clean up the resources

To delete all resources from your account, including the VPC, call the following command from the project root folder:

$ cdk destroy

As in the cdk deploy command, the destroy command attempts to run on the default profile defined in ~/.aws/config or ~/.aws/credentials. You can specify another profile by providing --profile as a command line option.

Conclusion

Although fine-grained access is now possible with the inclusion of Redis Role-Based Access Control (RBAC) users, user groups, and access strings in Amazon ElastiCache for Redis 6.x, there is no out-of-the box ability to associate ElastiCache RBAC users with IAM entities (roles, users, and groups).

This post presented a solution that restricted ElastiCache RBAC credentials (username and password) access by storing them in Secrets Manager and granting select IAM entities permissions to decrypt these credentials—effectively linking ElastiCache RBAC users with IAM roles by way of Secrets Manager as a proxy.

Additional benefits presented in this solution include:

  • ElastiCache RBAC passwords aren’t defined, stored, or shared in plaintext when ElastiCache RBAC users are created
  • ElastiCache RBAC users and groups can be defined wholly in AWS CDK (and by extension AWS CloudFormation) and included as infrastructure as code
  • You can trace Redis access to IAM users because ElastiCache RBAC credentials are stored and accessed through Secrets Manager and access to these credentials can be traced via AWS CloudTrail

From this post, you learned how to authorize access to specific ElastiCache Redis keys and commands through access strings. For more information about how to configure access strings, see Authenticating users with Role-Based Access Control (RBAC).

Beyond the scope of this blog post is the topic of secret rotation. It’s recommended to further secure your infrastructure by rotating ElastiCache RBAC user credentials stored in Secrets Manager on a periodic basis. In the our next blog post, we will demonstrate how to build on the architecture we presented here, by adding add a custom Lambda secret rotator function to the RedisRbacUser class.

About the authors

Claudio Sidi is a DevOps Cloud Architect at AWS Professional Services. His role consists of helping customers to achieve their business outcomes through the use of AWS services and DevOps technologies. Outside of work, Claudio loves going around the Bay Area to play soccer pickup games, and he also enjoys watching sport games on TV.

Jim Gallagher is an AWS ElastiCache Specialist Solutions Architect based in Austin, TX. He helps AWS customers across the world best leverage the power, simplicity, and beauty of Redis. Outside of work he enjoys exploring the Texas Hill Country with his wife and son.

Mirus Lu is a DevOps Cloud Architect at AWS Professional Services, where he helps customers innovate with AWS through operational and architectural solutions. When he’s not coding in Python, Typescript or Java, Mirus enjoys playing the guitar, working on his car and playing practical jokes on unsuspecting friends and coworkers.