AWS Cloud Financial Management
Better Together – Graviton 2 and GP3 with Amazon OpenSearch Service
Amazon OpenSearch Service is a managed service offering for the open-source OpenSearch project that makes it easy for you to perform interactive log analytics, real-time application monitoring and debugging, personalized search, and security information and event management (SIEM). If you manage Amazon OpenSearch Service domains and want to optimize their clusters to enhance performance and save money, this blog can provide some useful tips for you.
Amazon OpenSearch Service gives you the flexibility to choose from a variety of instance types to back Cluster master nodes and data nodes that are best suited for each workload. The available instance types are: general purpose, compute optimized, memory optimized, or storage optimized. With each new generation of instance type, Amazon OpenSearch Service continues to create a stronger price to performance ratio.
To further increase the price to performance capabilities that Amazon OpenSearch Service offers, Amazon OpenSearch Service supports AWS Graviton2 instances. Specifically, it supports: general purpose (M6G), compute optimized (C6g), memory optimized (R6g), and memory optimized with attached disk (R6gd) Graviton instances. From a performance perspective, these instances offer substantial improvements over previous generation instances, such as a 38% improvement in indexing throughput, 50% reduction in indexing latency, and a 40% improvement in query performance depending on the instance family and size when compared to the intel-based alternatives from the current generation (M5, C5, R5). AWS Graviton2 instances also include several performance optimizations such as larger caches per core, higher Amazon Elastic Block Store (Amazon EBS) throughput than comparable x86 instances, fully encrypted RAM, and more.
The technical advancements provided by Graviton2 based instances provide sufficient reason to provision and migrate your Amazon OpenSearch Service clusters to Graviton2 today. Performance advancements alone aren’t always enough to justify a transition from running OpenSearch on x86 based instances.
General Purpose SSD 3 (gp3) is the newest EBS based SSD volume available on AWS, offering better performance at a ~10% lower price point per GB when compared to previous generation gp2 volumes. Gp2 Volumes are still supported by Amazon OpenSearch Service, but new clusters should be provisioned with gp3 volumes for the best price/performance ratio. Gp2 volumes linearly scale IOPS performance based on the amount of storage configured, where larger volume sizes would allow for better throughput. Gp3 volumes are more consistent with throughput and provide a predictable 3,000 IOPS baseline performance and 125 MiB/s, regardless of volume size. In addition, OpenSearch Service provisions additional IOPS and throughput for higher volumes for optimal performance. When the application needs more performance, customers can scale up to 16,000 IOPS and 1,000 MB/s. More information on gp2 vs gp3 volumes can be found in the blog “Migrate your Amazon EBS volumes from gp2 to gp3 and save up to 20% on costs“.
This blog post will look at the cost savings you can recognize by running Amazon OpenSearch Service on Graviton2 based instances with gp3 based EBS storage, compared to running OpenSearch on traditional x86 based instances with gp2 based EBS storage.
Price Analyses of Running Amazon OpenSearch Service on Graviton2 Based Instances:
To demonstrate the cost savings of running Amazon OpenSearch Service on Graviton2 based Amazon EC2 instances, we will do a side-by-side comparison of two distinct Amazon OpenSearch Service deployment configurations. The first setup includes three r5.2xlarge.search instances for the master nodes, nine r5.2xlarge.search instances for the data nodes, and nine individual 1TB GP2 based EBS volumes. The second setup includes three r6g.2xlarge.search instances for the master nodes, nine r5.2xlarge.search instances for the data nodes, and nine individual 1TB GP3 based EBS volumes.
Figure 1. Cost Comparison Non-Graviton Vs Graviton2 based Amazon EC2 for OpenSearch
As you can see, the first setup, running x86 based instances and GP2 based EBS volumes would cost roughly $7,752.84 a month. Compare this to the $6,984.79 of the second setup, when it has the equivalent Amazon OpenSearch Service workload running on Graviton2 based instances with gp3 based EBS volumes. By making the change to Graviton2 based instances and gp3 based EBS volumes, you would save roughly 10% ($768.05/month or $9,216.60/year), while simultaneously achieving better performance.
Making the Switch to Graviton Based Clusters:
When it comes to identifying existing Amazon OpenSearch Service workloads that can benefit from running on Graviton2 based instances there is one primary requirement: Graviton based instances require Elasticsearch 7.9 or later, or any version of OpenSearch. The other key consideration is that Graviton2 based instances are only compatible with other Graviton instance types. You can’t combine Graviton and non-Graviton instances in the same cluster. If your Amazon OpenSearch Service workload is able to meet both requirements, then it is eligible to take advantage of the cost savings and performance benefits offered by Graviton.
To change the instance type and EBS volume type, you will need to perform a configuration change on your domain. Amazon OpenSearch Service uses a blue/green deployment process when updating domains. Blue/green refers to the practice of running two production environments, one live and one idle, and switching between the two environments as a software (or hardware) change Is made. In our example of moving an existing OpenSearch domain from r5.large.search instances to r6g.large.search instances, the blue/green deployment refers to the practice of creating a new environment for domain updates and routing new users to the new environment after those updates are completed. The blue/green deployment practice offers substantial advantages, as it minimizes downtime and maintains the original environment in the event that the deployment to the new environment is unsuccessful. For a detailed explanation of the configuration change, please refer to our documentation “Changes that usually cause blue/green deployments“.
To change the instance and volume types, first open the Amazon OpenSearch Service console. Then:
- On the left side menu, click on Domains.
- Check the box of the domain you’d like to change.
- Click on Actions, then click on “Edit Cluster Configuration”
Figure 2. Edit Cluster Configuration
From the Edit Cluster Configuration screen, you will need to change the instance type in two places, as well as the EBS volume type.
1. Data Nodes
- Instance type
- EBS volume type. Note: New clusters will already have gp3 as the default EBS volume type.
2. Cluster Manager Node
- Instance type
Figure 3. Edit Instance Type for Data Nodes
Figure 4. Edit Instance Type for Dedicated Master Nodes
Once you’ve made the changes, click the “Dry Run” button in the Summary section on the right-hand side of the screen and wait for the analysis to be completed.
Figure 5. Summary Dry Run
Once the dry run is completed, the results will be presented to you:
Figure 6. Dry Run Results
Click on “Save Changes” in the Summary section to begin the deployment process if no validation errors are present.
Figure 7. Summary Save Changes
Conclusion:
There are many benefits to running your Amazon OpenSearch Service workloads on Graviton2 based instances coupled with the gp3 EBS volume type. If you’re currently maintaining an Amazon OpenSearch Service workload, these changes are easy to make and can provide ~10% in savings with minimal effort. We recommend adopting or upgrading to 6-series, Graviton2-based instances for customers running all workloads on Amazon OpenSearch Service. We also strongly recommend testing your environment to determine the correct instance size and counts for the master and data nodes.
In the vast majority of cases, adopt Graviton2 instances for better price/performance for your Amazon OpenSearch Service domain. Increasing CPU utilization through higher concurrency and the shard: node ratio may increase the performance benefit you see from your Graviton2 instances, all while achieving ~10% in savings. Learn more best practices from this user guide “Operational best practices for Amazon OpenSearch Service”
If you have questions about this post, start a new thread on the Amazon OpenSearch Service forum or contact AWS Support.