Why is my Redis backup (.rdb) file always smaller than the BytesUsedForCache metric in the ElastiCache cluster?
Last updated: 2020-01-07
I'm using Amazon ElastiCache for Redis. Why is my Redis backup (.rdb) file always smaller than the BytesUsedForCache metric in the cluster?
The BytesUsedForCache metric includes bytes for all purposes, including actual key size, headers, and memory fragmentation. Expired keys also consume memory until Redis removes them asynchronously, either passively or actively. For more information, see Expire key seconds - How Redis expires keys on the Redis.io website.
ElastiCache uses both fork and fork-less backup processes. Both processes dump all keys on to the disk to create the .rdb file. The .rdb file doesn't use pointers or expired keys, and it doesn't deal with memory fragmentation, metadata, or buffers. This means that the backup files are smaller in size than the BytesUsedForCache metric. For example, your Redis cluster might show BytesUsedforCache as 15 GiB, and also show that the backup was initiated while there were heavy read/write requests on the cluster. After successful completion of the backup process, the backup cache size might be between 10 or 12 GiB, rather than 15 GiB, because the backup doesn't contain expired keys, pointers, and so on.