Posted On: Dec 14, 2022
Amazon EMR on EKS now supports accelerated computing over graphics processing unit (GPU) instance types using Nvidia RAPIDS Accelerator for Apache Spark. The growing adoption of artificial intelligence (AI) and machine learning (ML) in analytics has increased the need for processing data quickly and cost efficiently with GPUs. Nvidia RAPIDS Accelerator for Apache Spark helps customers leverage the benefit of GPU performance while saving infrastructure costs. With this release, EMR on EKS customer can use the RAPIDS accelerator by simply specifying the Spark-RAPIDS release label when calling EMR on EKS API.
Until now, EMR on EKS customers had to create a custom image to use Nvidia RAPIDS Accelerator. This requires engineering and test effort. In addition, with every new Nvidia RAPIDS release, bug fixes or security updates, customers had to rebuild the custom image and go through the testing again. Starting with EMR 6.9, EMR on EKS is introducing a new Nvidia RAPIDS Accelerator for Spark image. Customers can use the same StartJobRun API to run their Spark jobs, and simply specify a new Spark-RAPIDS release label to leverage RAPIDS Accelerator on an EKS cluster with GPU supported instance type.
This feature is available from Amazon EMR release 6.9 in all regions where Amazon EMR on EKS is available. To learn more about how to run Nvidia RAPIDS Accelerator for Apache Spark, please visit the documentation page.