Posted On: Jul 25, 2023
Amazon EMR Serverless is a serverless option that makes it simple for data analysts and engineers to run open-source big data analytics frameworks like Apache Spark and Apache Hive without configuring, managing, and scaling clusters or servers. Starting today, you can store logs for your EMR Serverless Spark and Hive applications in Amazon CloudWatch.
When you submit a job to an EMR Serverless application, you can decide where to store your application logs - in a managed storage system, an Amazon S3 location, or both. Now, you can choose Amazon CloudWatch as an option as well. This feature allows you to take advantage of CloudWatch log analysis features such as CloudWatch Logs Insights, Live Tail etc. as well as stream logs from CloudWatch to other systems such as Amazon OpenSearch for further analysis. When storing logs in Amazon CloudWatch, log data is always encrypted but you can choose to use your own encryption keys as well.
This feature is available for all release versions of EMR and in all regions where Amazon EMR Serverless is available. To learn more, see the EMR documentation.