Amazon EMR adds consistent view and server-side encryption support for Amazon S3

Posted on: Sep 17, 2014

The EMR File System (EMRFS) allows Amazon Elastic MapReduce (EMR) clusters to read from and write data to Amazon S3 securely and with high performance. EMRFS now supports consistent view, a feature that checks for list and read-after-write consistency for new objects in Amazon S3 by comparing EMRFS metadata with the results of Amazon S3 operations. EMRFS will retry Amazon S3 operations when needed, and you have the ability to define the retry logic. EMRFS also supports encryption for data at rest using Amazon S3 server-side encryption (SSE).

You can launch an Amazon EMR cluster with EMRFS consistent view and server-side encryption from the AWS Management Console, AWS CLI, bootstrap action, or SDK by selecting AMI 3.2.1. If you enable EMRFS consistent view, your Amazon EMR cluster will automatically create an associated EMRFS metadata store in Amazon DynamoDB, or you can specify an already existing metadata store. Also, multiple clusters can use the same metadata store and share a consistent view of S3. To learn more about EMRFS consistent view, click here. To learn more about EMRFS server-side encryption, click here.