Reducing Node Recovery Time by 95% Using Amazon EBS Multi-Attach with Teradata

Founded in 1979, Teradata is a software company that provides a complete database and cloud analytics solution to global enterprise customers in industries such as healthcare, banking, air travel, and more. These customers depend on high availability to run mission-critical workloads on their systems, and many of them are hesitant to migrate from on-premises systems to the cloud if reliability isn’t guaranteed. That’s why one of Teradata’s most important service-level agreements (SLAs) is 99.9 percent availability. “When nodes become unavailable, recovery time is paramount,” says Eric Shank, software architect at Teradata. “If a system is down for even 20 minutes, it can greatly impact our service.”

To meet the availability SLA for its data analytics solution, Teradata uses Amazon Elastic Block Store (Amazon EBS), high performance block storage at virtually any scale, to store thousands of volumes for each VantageCloud deployment. The company also uses Amazon EBS io2 Block Express, the highest-performance and most durable Amazon EBS volumes designed for critical applications at scale. This feature supports Amazon EBS Multi-Attach, which connects each volume to up to 16 instances on Amazon Elastic Compute Cloud (Amazon EC2)—a solution that offers secure and resizable compute capacity for virtually any workload—in the same Availability Zone. With the combination of the Multi-Attach feature and 99.999 percent durability provided by Amazon EBS io2 Block Express, attached compute resources with full read and write access to the shared volume are highly available and protected, so companies can prevent the loss of mission-critical data.

However, this alone doesn’t prevent data inconsistency if the file system can’t control access to a resource when using shared storage. This is especially relevant for larger systems, which many of Teradata’s customers use. To achieve its SLA of 99.9 percent availability, Teradata recognized the need for a nonvolatile memory express (NVMe) persistent reservations feature. NVMe reservations are a set of industry-standard storage fencing protocols that empower developers to prevent data inconsistency by failing over an application running on an Amazon EC2 instance to another instance connected to the shared Amazon EBS volume if the first fails, reducing downtime. Now, NVMe reservations form the backbone of Teradata’s multi-node clique architecture, which it specifically created for customers with large data stores. “NVMe reservations make a huge difference in availability,” says Shank.

When the NVMe reservations became available for the Multi-Attach feature on Amazon EBS io2 Block Express volumes, Teradata was one of the first AWS customers to implement it. “Teradata customers were waiting for this feature,” says Shank. “As soon as it launched, we immediately started recommending that customers use this configuration.”

Since using NVMe reservations supported by the Multi-Attach feature on Amazon EBS io2 Block Express volumes, Teradata can now easily switch nodes if an issue arises in the databases that it manages for its customers. As a result, it can reduce outage time from up to 20 minutes to 1 minute, an improvement of 95 percent. Immediately, Teradata’s customers benefited from better availability—up to 99.9 percent—and faster recovery time. “Using Amazon EBS Multi-Attach and NVMe reservations feature of Amazon EBS io2 Block Express, we can now give our customers greater confidence in the uptime of Teradata systems,” says Shank. “This confidence makes our customers more comfortable migrating their systems to the cloud.”

The addition of NVMe reservations supported by Amazon EBS Multi-Attach on io2 Block Express volumes also made it possible for Teradata to release an enhanced feature on its solution called multi-node clique (MNC), which increases resiliency for customers with large data stores. Using MNC, the software can coordinate access to storage safely through the I/O path instead of using APIs. When the software needs to replace a compute instance, it can do so faster than detaching and reattaching storage or copying data. “What’s great about NVMe reservations supported by Amazon EBS Multi-Attach is that they’re standardized, well documented, and well understood,” says Shank. “If a node fails, then we can revoke its reservation and prevent it from writing or destroying data. Because of that, we can now use MNC without worrying about data inconsistency.”

By simplifying the node failure recovery process using the Multi-Attach feature of Amazon EBS io2 Block Express volumes, Teradata is significantly lowering its system support costs, benefitting the company and its customers.

Teradata is planning to implement MNC for one of its large customers to take advantage of the improvements achieved through the company’s use of Amazon EBS io2 Block Express volumes with the Multi-Attach and NVMe reservations features. Moving forward, the company’s immediate goals are to bring these features to even larger systems and to bring more customers online. “This is a very new feature, but we expect great results,” says Omar Khan, senior product manager at Teradata.

Overall, Teradata is building a more efficient, more resilient system on AWS. “With the Multi-Attach and NVMe reservations features on Amazon EBS io2 Block Express, our system is a lot more reliable,” says Shank. “This is a huge deal for us and our customers.”