AWS Compute Blog
Using partition placement groups for large distributed and replicated workloads in Amazon EC2
This post is contributed by Ankit Jain – Sr. Product Manager, Amazon EC2 and Harsha Warrdhan Sharma – Global Account Solutions Architect at AWS
Before we introduced partition placement groups, customers deployed large distributed and replicated workloads across multiple Availability Zones to reduce correlated failures. This new Amazon EC2 placement strategy helps reduce the likelihood of correlated failures for workloads such as Hadoop, HBase, Cassandra, Kafka, and Aerospike running on EC2.
While Multi-AZ deployment offers high availability, some workloads are more sensitive to internode latency and could not be deployed across multiple zones. With partition placement groups, you can now deploy these workloads within a single zone and reduce the likelihood of correlated failures, improving your application performance and availability.
Placement group overview
Placement groups enable you to influence how your instances are placed on the underlying hardware. EC2 offers different types of placement groups to address different types of workloads. Use cluster placement groups that enable applications to achieve a low-latency network performance, necessary for tightly coupled node-to-node communication, typical of many HPC applications. Or, use spread placement groups to place a small number of critical instances on distinct racks, reducing correlated failures for your applications.
While spread placement groups work well for small workloads, customers wanted a way to reduce correlated failures for large distributed and replicated workloads that required hundreds of EC2 instances. They also wanted visibility into how instances are placed relative to each other on the underlying hardware. To address these needs, we introduced partition placement groups.
How partition placement groups work
To isolate the impact of hardware faults, EC2 subdivides each partition placement group, into logical segments called partitions. EC2 ensures that no two partitions within a placement group share the same racks. The following diagram shows a partition placement group in a single zone:
Applications such as Hadoop, HBase, Cassandra, Kafka, and Aerospike have replicated nodes for fault tolerance and use the topology information to make intelligent data storage decisions. With partition placement groups, EC2 can place the replicated nodes across separate racks in a zone and isolate the risk of hardware failure to only one node. In addition, partition placement groups offer visibility into the partitions that allows these rack-aware applications to make intelligent data replication decisions, increasing data availability and durability.
Deploying applications using partition placement groups
Here’s an example of how to use a partition placement group through the AWS Management Console or AWS CLI.
First, in the console, choose EC2, Placement Group, create a partition placement group, and then define the number of partitions within that placement group. When creating a partition placement group, we recommend mirroring the replication factor of your application and the number of partitions that you run in that placement group. For instance, if a Hadoop cluster has a replication factor of 3, then you should define a partition placement group with three partitions.
You can also run the following AWS CLI command to perform the same action:
After defining the placement group, launch the desired number of EC2 instances into this placement group from the EC2 launch instance wizard.
If you specify Auto distribution for the Target partition field at the time of launch, EC2 tries to evenly distribute all instances across the partitions. For example, a node might consist of 100 EC2 instances and there could be three such replicated nodes in your application. You can launch 300 EC2 instances, select the partition placement group, and keep Auto distribution as the default. Each partition then has approximately 100 instances, with each partition running on a separate group of racks.
Run the following command (auto distribution is the default behavior):
Alternatively, you can specify a target partition number at launch. As your workload grows, this feature makes it easy to maintain the placement group. You can replace instances in a specified partition or add more instances to a specified partition, using the Target partition field.
Run the following command and specify PartitionNumber:
After you launch the EC2 instances, you can view the partition number associated with each instance. For these rack-aware applications, you can now treat the partition number associated with instances as rack-ids and pass this information to the application. The application can then use this topology information to decide the replication model to improve the application performance and data reliability. View this information in the console by filtering by placement group name or by the partition number.
Run the following command:
To place the instance in a different AZ but as part of the same partition placement group, select an alternate subnet for the instance during launch.
Conclusion
Partition placement groups provide you with a way to reduce the likelihood of correlated failures for large workloads. This allows you to run applications like Hadoop, HBase, Cassandra, Kafka, and Aerospike within a single Availability Zone. By deploying these applications within a single zone, you can benefit from the lower latency offered within a zone, improving the application performance. Partition placement groups also offer elasticity, allowing you to run any number of instances within the placement group. For more information, see Partition Placement Groups.