How does elastic resize work in Amazon Redshift?

Last updated: 2022-10-31

I want to perform an elastic resize on my Amazon Redshift cluster. How does elastic resize work and what are some considerations for using elastic resize?

Short description

Amazon Redshift allows you to migrate to a certain number of nodes during a cluster resize. By default, Amazon Redshift aims to maintain the same number of data slices in the target cluster. Slice mapping reduces the time required to resize a cluster. During the slice-mapping process, Amazon Redshift reshuffles the cluster data slices to the new compute nodes in your target cluster.

For classic resize, all rows are copied to the cluster, mapping them to a slice based on distribution settings. For elastic resize, you can copy the whole slice of data to a node where the slice is mapped.

If you plan to perform an elastic resize on your Amazon Redshift cluster, consider the following:

  • Elastic resize doesn't sort tables or reclaim disk space. Run VACUUM to sort tables and reclaim disk space.
  • Elastic resize is available only for Amazon Redshift clusters that use the EC2-VPC platform.
  • Elastic resize often requires less time to complete than a classic resize. To compare, classic resize operations provision a new cluster while copying over data from your source cluster. The classic resize operation first distributes the data to the new nodes according to distribution style. Then, it runs the ANALYZE command to update table statistics. This means that a classic resize operation takes more time to complete than an elastic resize operation on your Amazon Redshift cluster.

Resolution

How elastic resize works

In Amazon Redshift, elastic resize can work differently depending on the target node type. Check whether the target node type is the same as the source node type.

To check your node type, sign in to the Amazon Redshift console. From the navigation menu, choose Clusters. The Clusters page indicates the node type under each cluster name. Or, you can use the describe-clusters AWS Command Line Interface (AWS CLI) command to obtain more information about your Amazon Redshift cluster:

aws redshift describe-clusters --region <Cluster Region>

Example 1: Target node type is same as the existing node type

Amazon Redshift automatically redistributes data to new nodes when you resize a cluster using elastic resize (add or remove nodes without changing the node type).
Unlike classic resize (which provisions a new cluster and transfers data to it), elastic resize doesn't create a new cluster. Elastic resize typically completes within a few minutes. You can expect a small increase in your query runtime while elastic resize completes data redistribution in the background.

Note: Your Amazon Redshift cluster is temporarily unavailable for a few minutes during the metadata migration. For more information about the Amazon Redshift elastic resize process, see Elastic resize.

Example 2: Target node type is different from the existing node type

If your node type changed, Amazon Redshift creates a snapshot first. A new target cluster is then provisioned with the latest data from the snapshot, and data is transferred to the new cluster in the background. During the data transfer, your Amazon Redshift cluster operates in read-only mode and all writes are blocked. When the resize nears completion, Amazon Redshift automatically updates the new cluster’s endpoint to match your existing cluster’s endpoint. All connections to the original cluster are then closed.

DC2 and DS2 node count limitations

If you perform an elastic resize on your Amazon Redshift cluster, then note the following limitations for DC2 and DS2 node types:

  • For dc2.large or ds2.xlarge node types, use half or double the current number of nodes. For example, a cluster with 6 nodes can be resized to either 12 nodes or 3 nodes.
  • For dc2.8xlarge, ds2.8xlarge, or ra3.xlplus node types, use half or up to double the number of nodes. For example, a cluster with 6 nodes can be resized to 3, 4, 5, 7, 8, 9, 10, 11, 12 nodes.
  • For ra3.16xlarge or ra3.4xlarge node types, use quarter or up to four times the current number of nodes. For example, a ra3 cluster with 8 nodes can be resized to 2, 3, 4, 5, 6, 7, 9 up to 32 nodes.

Elastic resize best practices

Consider the following best practices when performing an elastic resize on your Amazon Redshift cluster:

Here's an example of how to use the describe-node-configuration-options command:

aws redshift describe-node-configuration-options --action-type resize-cluster --cluster-identifier <Cluster Name> --region <Cluster Region>

Here's an example of how to use the resize-cluster command:

aws redshift resize-cluster --cluster-identifier  <Cluster Name> --cluster-type multi-node --node-type <Target Node Type> --number-of-nodes <Number of Target Nodes> --no-classic —region <Cluster Region>

Additional considerations

Review these additional considerations when performing an elastic resize on your Amazon Redshift cluster:

  • A cluster snapshot is required for an elastic resize. You can manage snapshots using the console or the Amazon Redshift CLI and API.
    Note: If you receive an error when running the AWS CLI command, be sure to use the most recent version of the CLI.
  • After you initiate an elastic resize operation in Amazon Redshift, the operation can't be canceled. Wait until the resize operation completes before performing another resize operation or cluster reboot.
  • The new node configuration must have enough storage for existing data. Even when you add nodes, your new configuration might not have enough storage because of the way that the data is redistributed. For more information about storage space, see Why does a table in an Amazon Redshift cluster consume more or less disk storage space than expected?
  • Performing an elastic resize on your Amazon Redshift cluster can cause data skew between nodes from an uneven distribution of data slices. If you observe data skew in your Amazon Redshift cluster, perform a classic resize instead.
  • Perform a classic resize on your Amazon Redshift cluster if it's the best option for your use case. For example, you can perform a classic resize if you're resizing to a single-node cluster. An elastic resize allows you to add or remove nodes from the cluster by preserving the original configuration's slice count. However, it can introduce performance variation. If you want your node slices to match the number of slices in your target node type, use a classic resize. For more information, see Resizing clusters in Amazon Redshift.
  • When an elastic resize is started and a snapshot operation is underway, the resize can fail if the snapshot doesn’t complete within a few minutes.
  • Elastic resize can't be used to resize from or to single node clusters.
  • For reserved nodes, such as DS2 reserved nodes, you can upgrade to RA3 reserved nodes when you perform a resize. You can upgrade the node when you perform an elastic resize or use the console to restore from a snapshot.