How does elastic resize work in Amazon Redshift?
Last updated: 2020-10-21
I want to perform an elastic resize on my Amazon Redshift cluster. How does elastic resize work and what are some considerations for using elastic resize?
Amazon Redshift allows you to migrate to a certain number of nodes during a cluster resize. By default, Amazon Redshift aims to maintain the same number of slices in the target cluster. With slice mapping, the time required to resize a cluster is reduced. During the slice mapping process, Amazon Redshift reshuffles the cluster slices to the new compute nodes in your target cluster.
If you plan to perform an elastic resize on your Amazon Redshift cluster, consider the following:
- Elastic resize doesn't sort tables or reclaim disk space. Run VACUUM to sort tables and reclaim disk space.
- Elastic resize is only available for Amazon Redshift clusters that use the EC2-VPC platform.
- Elastic resize operations often require less time to complete than classic resize operations. Elastic resize operations do not sort tables or reclaim disk space. To compare, classic resize operations provision a new cluster, while copying over data from your source cluster. The classic resize operation then distributes the data to the new nodes according to distribution style, and runs the ANALYZE command to update this information. As a result, it can take longer to perform a classic resize than an elastic resize on your Amazon Redshift cluster.
How elastic resize works
In Amazon Redshift, elastic resize can work differently depending on the target node type. Check whether the target node type is the same as the source node type.
To check your node type, sign in to the Amazon Redshift console. From the navigation menu, choose Clusters. The Clusters page indicates the node type under each cluster name. Or, you can use the describe-clusters AWS Command Line Interface (AWS CLI) command to obtain more information about your Amazon Redshift cluster:
aws redshift describe-clusters --region <Cluster Region>
Example 1: Target node type is same as the existing node type
When we resize a cluster in Amazon Redshift using elastic resize (without changing the node type), Amazon Redshift automatically redistributes data to the new nodes. Unlike classic resize (which provisions a new cluster and transfers data to it), elastic resize does not create a new cluster. Elastic resize operations typically complete within a few minutes. You can expect a small increase in your query execution time while elastic resize completes data redistribution in the background.
Example 2: Target node type is different from the existing node type
If your node type has changed, Amazon Redshift creates a new cluster and copies data to this cluster from the latest snapshot. During the data transfer, your Amazon Redshift cluster operates in read-only mode and all writes are blocked. After the resize operation completes, Amazon Redshift automatically updates the new cluster’s endpoint to match your existing cluster’s endpoint. All connections to the original cluster are then terminated.
DC2 and DS2 node count recommendations
If you perform an elastic resize on your Amazon Redshift cluster, note the following recommendations for DC2 and DS2 node types:
- For dc2.large or ds2.xlarge node types, use half or double the current number of nodes. For example, a 6-node cluster can be resized to either 12 nodes or 6 nodes.
- For dc2.8xlarge, ds2.8xlarge, ra3.4xlarge, or ra3.16xlarge node types, use half to double the current number of nodes. For example, a 6-node cluster can be resized to 3, 4, 5, 7, 8, 9, 10, 11, or 12 nodes.
Elastic resize best practices
Consider the following best practices when performing an elastic resize on your Amazon Redshift cluster:
- Take a snapshot before you start the elastic resize process, especially if you are resizing a new cluster.
- Use the describe-node-configuration-options AWS CLI command to get possible node configurations for a resize operation.
Note: If you receive an error when running your AWS CLI command, be sure that you’re using the most recent version of the CLI.
- VACUUM the cluster before resizing. Elastic resize does not automatically delete rows that are marked for deletion.
- Use the resize-cluster command to resize your cluster.
Here's an example of how to use the describe-node-configuration-options command:
aws redshift describe-node-configuration-options --action-type resize-cluster --cluster <Cluster Name> --region <Cluster Region>
Here's an example of how to use the resize-cluster command:
aws redshift resize-cluster --cluster-identifier <Cluster Name> --cluster-type multi-node --node-type <Target Node Type> --number-of-nodes <Number of Target Nodes> --no-classic —region <Cluster Region>
Review these additional considerations when performing an elastic resize on your Amazon Redshift cluster:
- A cluster snapshot is required for an elastic resize. You can manage snapshots using the console or the Amazon Redshift CLI and API.
Note: If you receive an error when running the AWS CLI command, be sure to use the most recent version of the CLI.
- After you initiate an elastic resize operation in Amazon Redshift, the operation cannot be cancelled. Wait until the resize operation completes before performing another resize operation or cluster reboot.
- The new node configuration must have enough storage for existing data. Even when you add nodes, your new configuration might not have enough storage because of the way that the data is redistributed. For more information about storage space, see Why does a table in an Amazon Redshift cluster consume more or less disk storage space than expected?
- When you perform an elastic resize on your Amazon Redshift cluster, it can cause data skew between nodes from an uneven distribution of slices. If you observe data skew in your Amazon Redshift cluster, perform a classic resize instead.
- Perform a classic resize on your Amazon Redshift cluster if it is the best option for your use case. For example, you can only perform a classic resize if you are resizing to a single-node cluster. Or, perform a classic resize if you want your node slices to match the number of slices in your target node type.