ElastiCache for Redis Backup and Restore with Cluster Resizing

Clusters provide a lot of capabilities in Amazon ElastiCache for Redis. Using autosharding to distribute data across the shards of a cluster enables high performance (over 4.5m writes/sec and 20m reads/sec, with each operation typically under 500µs) and high capacity (up to 3.5TiB of data) with high reliability and with recovery of failed nodes usually within 30 seconds.

When you create an ElastiCache cluster, you set the number of shards in the cluster. Each shard has one primary node (for reads and writes) and from zero to five replica nodes (for reads and failover protection). A cluster can be as small as a single shard with zero replicas (1 node) and as large as 15 shards each with 5 replicas (90 total nodes). You also choose the instance type that will be used by all the nodes in your cluster, which range from the tiny cache.t2.micro (0.555 GiB of memory) through the m3/m4 family, to the huge cache.r3.8xlarge (237 GiB).

But what happens if you need to change these choices? How can you change to a larger or smaller number of shards or to a different instance type?

Fortunately, ElastiCache makes this easy. All you need to do is take a backup.

Backups, which are comprised of point in time snapshots of individual shards, are the backup and restore tool for ElastiCache. You can create a backup of your cluster at any time using the AWS Console or AWS CLI.

Once the backup is created, you can restore it to any ElastiCache for Redis cluster, as long as the version of the cluster is at least as current as that of the backup and the total memory in the cluster is large enough to contain the Redis keyspace stored within the backup.

So, for example, if you started with a 3-shard cluster and want to expand it to a 5-shard cluster, just:

1. Backup the 3-shard cluster
2. Restore the backup into a new 5-shard cluster

You can use this technique to easily test different cluster configurations to figure out the best combination of shards and instance types for your needs.

You can also restore snapshots taken from non-ElastiCache for Redis clusters, if you’re using Redis off the Cloud or running it yourself on EC2. Just copy your snapshots to S3, and then use them with ElastiCache Import. ElastiCache will parse the key space within the specified snapshots and restore it into an ElastiCache for Redis Cluster with the number of shards and Redis slot ranges of your choice.

Try it out!