AWS Partner Network (APN) Blog

Protecting Apache Iceberg Lakehouses at Scale with Rubrik on AWS

By: Sufyan Barkat, Sr. Partner Solutions Architect – AWS
By: Brian Mislavsky, Sr. Advisory Alliance Solutions Architect – Rubrik

Rubrik Logo
Rubrik Connect Button

Many organizations running production analytics and artificial intelligence (AI) on Amazon Web Services (AWS) have standardized on Apache Iceberg as their open table format. AWS-native services like AWS Glue, Amazon Simple Storage Service (Amazon S3), and Amazon S3 Tables have simplified managing Iceberg tables at scale. As these tables have become more central to the business, a gap has become apparent. Recovering an Iceberg table after a data loss event is more involved than recovering the files it is made of. Copying the underlying files back is comparatively straightforward. Restoring a table that is in fact queryable, with its metadata, manifests, and catalog all consistent, is not. This matters more than it used to, because the data behind these tables now serves as the input layer for machine learning (ML) model training, real-time decisioning, and financial reporting. When that data becomes unavailable, the impact extends well beyond storage.

In this post, we discuss why the data lakehouse has become a protection surface in its own right, why Apache Iceberg’s built-in tools were not designed to serve as an independent recovery layer, and how Rubrik, an AWS Partner, addresses this with Apache Iceberg Data Protection built natively on AWS.

The lakehouse is a new protection category

For years, enterprise backup solutions focused on virtual machines, databases, and file systems. Over time, this has extended to cloud services such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Relational Database Service (Amazon RDS). The data lakehouse does not fit neatly into any of these categories. An Iceberg table is not a single object that you can copy and restore. It is a coordinated set of four layers that must remain in sync: data files, metadata files, manifest lists, and the catalog that ties them together.

This distinction is important. A backup that copies only the files does not give you a working table on recovery. It gives you an archive that still needs to be reassembled, often manually, before the table can be queried again. Recovering files is not the same as recovering a working table, and that difference is what makes lakehouse protection a distinct requirement rather than a feature added to an existing product.

Apache Iceberg’s native time travel capability is often assumed to address this need. Time travel is a useful feature for operational rollbacks and query flexibility. However, it operates inside the same account as the data it rolls back. If that account is subject to broad or unauthorized access, time travel is affected along with everything else.

Three trends converging on AWS

Several forces make this a timely consideration for AWS customers.

Iceberg adoption is growing rapidly on AWS. The format’s ACID compliance, schema evolution, and engine-agnostic design have made it a common foundation for lakehouse architecture. AWS Glue and Amazon S3 Tables have accelerated that adoption by offering managed catalog services that handle Iceberg at scale, lowering the barrier to running it in production.

AI has changed the risk profile of lakehouse data. Iceberg tables used to hold analytics output, such as reports and dashboards. Now they often hold the input: training data for ML models, feature stores that drive real-time decisions, and source-of-truth datasets for regulatory and financial reporting. A loss event that once delayed a dashboard can now require model retraining, stall production pipelines, or create compliance exposure.

Purpose-built lakehouse protection is the next area of focus for cyber resilience. As enterprises formalize this as part of their broader resilience strategy, it takes more than a rollback feature: independent storage outside the primary environment, catalog-aware recovery that returns a fully working table, and policy-based governance across the data estate.

When time travel isn’t enough

In a representative scenario, a data engineering team at a financial services firm runs Iceberg workloads on AWS. Their tables feed ML model training, business intelligence dashboards, and the quarterly financial close. An unauthorized user gains access to credentials in a ransomware event and deletes the underlying Amazon S3 data files.

The team turns to time travel, the same capability they rely on for routine rollbacks, and finds that it points to data that no longer exists, within the same account that was affected. Even if the underlying files can be recovered, the table is still not queryable, because the catalog was never updated to reflect the restored state. The table state must then be reconstructed manually, a process that can take days for a multi-petabyte table, delaying every downstream process that depends on it.

With Rubrik, the outcome is different. Rubrik’s backups can live in the customer’s own AWS account, protected with Amazon S3 Object Lock or logically air-gapped in Rubrik Cloud Vault, so an unauthorized user with those same credentials is prevented from altering or deleting them. On recovery, Rubrik updates the catalog automatically, and the table returns to a complete queryable state without a manual rebuild.

How Rubrik protects Apache Iceberg on AWS

Apache Iceberg Data Protection is not a standalone product. It extends the Amazon S3 data protection Rubrik already provides, adding an Iceberg-aware layer to the Rubrik solution customers already use to protect the rest of their AWS estate. Five characteristics define the approach:

Iceberg-aware backup and recovery. Rubrik protects the full table, not only its contents: Amazon S3 data files, metadata files, manifest lists, and the catalog. This includes self-managed Iceberg tables in customer Amazon S3 buckets cataloged in AWS Glue Data Catalog, as well as AWS-managed tables in Amazon S3 table buckets. Recovery is atomic, meaning a table is restored as a consistent, queryable unit rather than in a partial state. Because the data, metadata, and catalog are restored together, teams skip the manual table-state reconstruction that can stretch to days at multi-petabyte scale and can bring tables back in hours instead. For in-place restores, only the changed data needs to move, so recovery is faster and does not require rehydrating the entire table. Restores can be scoped to a single table, directed to a new or existing table, and run across accounts and Regions.

A choice of where backups are stored. Customers choose where their backups live. They can store backups in an Amazon S3 bucket in their own AWS account, where Rubrik enables Amazon S3 Object Lock in Compliance mode. Backups remain immutable and safe even if administrative credentials in that account are subject to unauthorized access. Alternatively, customers can use Rubrik Cloud Vault, a completely isolated, vendor-managed vault within Rubrik’s tenant. Customers with data-residency, sovereignty, or contractual requirements can keep backups in their own account, while those who prefer to offload that responsibility entirely can use Rubrik Cloud Vault.

Cost optimization at petabyte scale. Rather than copying every snapshot in a table’s history, Rubrik protects only the latest clean, compacted snapshot. This reduces redundant or deleted files and shortens backup windows, while still preserving the ability to recover a fully consistent table, which is what matters most for cyber recovery. Backup data can be written directly to warm or cold Amazon S3 Storage Classes so that customers do not pay active-storage rates for data they access infrequently, which adds up quickly across a multi-petabyte lake.

A single platform across the AWS environment. Iceberg protection runs on the same Rubrik solution that covers Amazon EC2, Amazon RDS, Amazon DynamoDB, and Amazon S3. This provides consistent policies, role-based access controls, and compliance reporting in one place.

Lower-risk recovery through branching. Rubrik can recover a table to an isolated branch, where teams validate the data and schema before promoting it to production, so recovery testing never runs against live tables.

The architecture in Figure 1 reflects these principles. Rubrik Security Cloud authenticates through a scoped cross-account role that uses temporary AWS Security Token Service (AWS STS) credentials instead of long-lived keys to reach the catalog and tables in the customer’s account. The snapshot processing runs on a Rubrik Exocompute instance inside that account before the backup is written to whichever destination the customer has chosen.

Figure 1. Rubrik Apache Iceberg Data Protection architecture on AWS

Figure 1 – Rubrik Apache Iceberg Data Protection architecture on AWS

Conclusion

In this post, we looked at why the modern data lakehouse now needs protection of its own. Apache Iceberg’s native time travel was not built to protect against unauthorized access to an account. Apache Iceberg Data Protection closes that gap. It backs up the full table across both self-managed Amazon S3 buckets and Amazon S3 table buckets, and stores backups in the customer’s own AWS account or in Rubrik Cloud Vault. As enterprises build AI and analytics pipelines on Iceberg, catalog-aware and independently stored backup is becoming a foundational requirement.

To learn more, read the Rubrik Apache Iceberg Data Protection datasheet, watch the interactive demo, or find Rubrik in AWS Marketplace to get started.

Connect with Rubrik


Rubrik – AWS Partner Spotlight

Rubrik, the Security & AI Operations Company, is an AWS Partner with the AWS Storage ISV Competency and AWS Resilience Software Competency and is at the Validated stage of the AWS Partner Path. Rubrik leads at the intersection of data protection, cyber resilience, and enterprise AI acceleration. It protects enterprise data across cloud, on-premises, and SaaS environments and is listed on AWS Marketplace.

Contact Rubrik | Partner Overview | AWS Marketplace