Connect your Jupyter notebooks to Amazon EMR Serverless using Apache Livy endpoints
Today, we are excited to announce that Amazon EMR Serverless now supports endpoints for Apache Livy. Customers can now securely connect their Jupyter notebooks and manage Apache Spark workloads using Livy’s REST interface.
Amazon EMR Serverless is a serverless option in Amazon EMR that makes it simple and cost effective for data engineers and analysts to run petabyte-scale data analytics in the cloud. With the Livy endpoints, setting up a connection is easy - just point your Livy client in your on-premises notebook running Sparkmagic kernels to the EMR Serverless endpoint URL. You can now interactively query, explore and visualize data, and run Spark workloads using Jupyter notebooks without having to manage clusters or servers. In addition, you can use the Livy REST APIs for use cases that need interactive code execution outside notebooks.
This feature is generally available on EMR release versions 6.14 and later and in the following AWS Regions:
US East (Ohio, N. Virginia), US West (N. California, Oregon), Canada (Central), Asia Pacific (Hong Kong, Jakarta, Mumbai, Osaka, Seoul, Singapore, Sydney, Tokyo), Europe (Frankfurt, Ireland, London, Milan, Paris, Spain, Stockholm), Middle East (Bahrain, UAE), Africa (Cape Town), and South America (São Paulo). To learn more, see the EMR documentation.