Posted On: Nov 17, 2023
We are excited to announce general availability of AWS Lake Formation based fine-grained access controls (FGAC) on Open Table Formats (OTFs) with Amazon EMR on EC2 clusters. With today’s launch, Amazon EMR simplifies security and governance over transactional data lakes by providing access controls at table, column and row level permissions with your Apache Spark jobs accessing Apache Iceberg, Apache Hudi and Delta tables.
Customers use OTF tables to manage continuously evolving data sets while maintaining query performance. They need a way to administer granular access permissions for these OTF tables for different users, business units, orgs at scale. With this launch, customers can define granular permissions in Lake Formation for OTF tables and apply them when running data processing jobs via Spark on Amazon EMR clusters. They also get read, and write (inserts) access to OTF tables and can use features such as running snapshot queries to get the latest snapshot of the table at a given commit or compaction instant, incremental, time-travel, and DML queries.
This feature is available with Amazon EMR release 6.15 for Amazon EMR on EC2 clusters in all regions where Amazon EMR is available. To learn more, please visit “Integrate Amazon EMR with AWS Lake Formation” section in the documentation.