Redis Cluster 101

Learn more about Redis Cluster!

What is Redis Cluster?

While Redis Cluster is technically a feature of Redis, it feels more like an extremely close relative. OK, it sounds a bit exaggerated, but starting with that distinction is the best possible mindset for understanding what Redis Cluster is all about.

The obvious distinction is the fact that Redis Cluster operates as a network of masters and replicas, and this network heals itself whenever possible: if a master goes down and a suitable replacement is found among the replicas, the cluster will trigger a failover and the chosen replica will take the role of its master. And the obvious advantage of this setup is the fact that you can spread your data across different servers to benefit from higher availability and scale as you need. 

Hash Slots

In order to understand other differences between standalone Redis and Redis Cluster, you have to learn about hash slots, or buckets as we call them just to explain what they are and how they work. Imagine there are four buckets, and a hash function that maps any key —any string— to any of those four buckets. Let’s say "foo" maps to bucket 3, "bar" maps to bucket 1, "baz" maps to bucket 1 again, "qux" maps to bucket four, and so on. What is important here is that a given string will always map to the same bucket. The hashing function will produce a more or less even distribution, so if you produce random keys and you map them to four buckets, those buckets will end up with a similar amount of keys. Now imagine that a Redis server can handle either one, two, three or four buckets. If it handles all four buckets, then we could expect it to behave like standalone Redis, because every key will be accounted for and all keys will live on the same server. (Conceptually, you can think about standalone Redis as having one big bucket that fits all possible keys.) If we have four buckets, we can make different servers handle different buckets under two conditions: first, a bucket can only be assigned to one server; second, all of the buckets must be taken care of. It doesn’t matter if you assign buckets one, two and three to one server and bucket four to another, or if you set up four servers and assign one bucket to each. As long as those two conditions are met, the setup is correct.

Continuing with our four buckets example: how does the fact that keys live in different buckets affect the behavior of Redis? The rule is that if all the keys mentioned in a command live in the same bucket, then nothing changes from what you would expect in standalone Redis. But if you are sending a command that operates on multiple keys and those keys live on different buckets, then what you get is an error. Don’t lose your hope yet, because you are not completely at the mercy of that hashing function: Redis Cluster lets you cheat and decide in advance in which bucket a given key will live, and that’s enough for you to group your keys in a meaningful way. Later, we will go back to this point to see how to instruct Redis Cluster to group some keys according to your needs.

In Redis Cluster, those buckets are called hash slots, and there are 16384 of them. Or, if you prefer round hexadecimal numbers, 0x4000. The same logic applies: you can distribute those hash slots among your Redis Cluster servers in any fashion you like, as long as all the hash slots are covered and as long as each individual hash slot is handled by one and only one server. You can assign the hash slots to the different servers one by one, but there are tools to help you with that process. In any case, when reasoning about Redis Cluster it is useful to use a conceptual model of a few buckets (four, six, eight), so that you can understand what is going on without overloading your brain with the complexity of so many hash slots.

What we have so far: a Redis Cluster is a network of interconnected masters and replicas. Each master is in charge of one or more hash slots, and each hash slot can have zero or more keys depending on the usage pattern. All hash slots must be accounted for by the masters. Each master can have zero or more replicas. Each replica will have an exact copy of the hash slots handled by its corresponding master. If a master goes down, the surviving cluster nodes communicate and elect a suitable replacement. If that master has no replicas, then no failover is possible. That’s why a sane setup includes at least one replica per master.

Before closing the chapter on hash slots, let’s talk about an advanced feature that lets you group keys in the same hash slot. It was mentioned that Redis Cluster uses a hashing algorithm to determine the hash slot a given key belongs to. Let’s say your application stores sets under the keys "user:1:friends" and "user:2:friends", and you want to intersect those sets to obtain a set of friends common to those two users. If Redis Cluster decides to send one set to one hash slot and the other set to another hash slot, then you won’t be able to use a command to intersect them. The advanced feature is called key hash tag, and it works like this: if instead of using "user:1:friends" you use "{foo}user:1:friends", then Redis Cluster will only use the substring "foo" in order to compute the hash. So all keys with they hash tag {foo} will end up in the same hash slot, and you will be able to run multi-key operations on those keys.

Clients and Redirections

If the keys live in hash slots, and different hash slots are handled by different servers, how do clients know where to connect if they want to retrieve the value associated with a given key? The short answer is: they don’t. Clients don’t have to know which server is handling a given key. Imagine a simple command like GET foo. Innocent as it looks, it can trigger a somewhat complex reply. When a client sends that command to a server, if the server is in charge of that key, then it will reply with the corresponding value. If the server is not in charge of the key, it will reply with a MOVED error message. Here’s an example MOVED error message:

-MOVED 1234 127.0.0.1:6381

Let’s unpack it:

-MOVED is the error message.

1234 is the hash slot where the key is located.

127.0.0.1:6381 is the host and port of the server in charge of the hash slot.

With that information, the client can connect to the server running on 127.0.0.1:6381 and retry the command. There are some edge cases that will be covered in future blog posts, but this basic scenario is what you will encounter most of the time.

There are many open source Redis clients that integrate that logic so that you don’t have to handle the redirection by hand. Some clients even keep an internal map of where each hash slot is located, so that when you need to send a command the client determines to which server it should be dispatched. The advantage of using that kind of client is that the performance you will get from any given server in your cluster will be almost the same to what you would get from a standalone Redis server. But if you consider that Redis Cluster allows you to scale to many servers and it gives you some high availability guarantees, then the performance over all the cluster will scale almost linearly.

There's one important caveat: if a client connects to a replica and sends the command GET foo, the reply will be a MOVED error message even if that replica is in charge of the hash slot where the key "foo" is located. That behavior can be changed by sending first the command READONLY, which instruct the replica to serve the client's request even if the data is stale. The READWRITE command can be used to restore the connection to the original state.

Conclusion

Redis Cluster has some notable advantages over standalone Redis in terms of scalability and availability. On the downside, it's a bit harder to operate because of the inherent complexity of distributed systems. With the help of some simple analogies, it is possible to profit from all the advantages of Redis Cluster and scale your database with confidence. In the upcoming blog posts, we will setup a cluster and we will get familiar with how it works and how you can use it in your projects. Remember that ElastiCache makes it extremely easy to create and configure your Redis Cluster. Stay tuned.

Related articles

What is Redis?

Redis Blog Series

Blog: What is New with Redis 5.0

Blog: Working with Redis Streams

Blog: Redis Streams and Message Queues

Coming up: Redis Cluster Blog Series

Redis Cluster Blog 1: Redis Cluster 101

 

Fully managed Redis on AWS

Amazon offers a fully managed Redis service, Amazon ElastiCache for Redis, available for trial at no cost with the AWS Free Tier. Amazon ElastiCache for Redis makes it easy to set up, operate, and scale Redis deployments in the cloud. With Amazon ElastiCache, you can deploy internet-scale Redis deployments in minutes, with cost-efficient and resizable hardware capacity.

Get started with free Amazon ElastiCache for Redis in three easy steps:

Amazon Elasticache for Redis

Sign up

Get access to the Amazon ElastiCache Free Tier.
Learn Amazon Elasticache for Redis

Learn with simple tutorials

Explore how to create a Redis cluster.
Get Started with ElastiCache for Redis

Start building

Begin building with help from the user guide.