AWS Big Data Blog
Author: Philippe Wanner
Philippe Wanner is a Tech Lead at AWS. His role is to spread the best practices for large organizations and to drive impact through his talented team. His expertise and leadership focus on the intersection of distributed systems, data, AI, and business transformation.
Modernize Apache Spark workflows using Spark Connect on Amazon EMR on Amazon EC2
In this post, we demonstrate how to implement Apache Spark Connect on Amazon EMR on Amazon Elastic Compute Cloud (Amazon EC2) to build decoupled data processing applications. We show how to set up and configure Spark Connect securely, so you can develop and test Spark applications locally while executing them on remote Amazon EMR clusters.