Amazon Web Services

In this AWS re:Invent 2023 session, Netflix engineers Ashwin Kayyoor and Rakesh Veeramacheneni share their journey of modernizing Netflix's massive data lake using Apache Iceberg. They discuss the challenges of managing an exabyte-scale data warehouse and the transition from a Hive-based system to an Iceberg-only architecture. The presentation covers the development of custom tooling, ecosystem services, and unique features like secure Iceberg tables and the Iceberg REST catalog. The speakers detail the migration process, including strategies to minimize data movement and user friction while ensuring business continuity. They also highlight the benefits of Iceberg, such as ACID transactions, rich metadata layers, and improved query performance. The talk provides valuable insights for organizations looking to scale their data infrastructure and leverage open-source table formats for efficient data management.

customer-stories
product-information
skills-and-how-to
media-and-entertainment
data
Show 7 more

Up Next

VideoThumbnail
40:23

Set Up and Use Apache Iceberg Tables on Your Data Lake - AWS Virtual Workshop

Nov 22, 2024
VideoThumbnail
6:45

Grindr's Next-Gen Chat System: Leveraging AWS for Massive Scale and Security

Nov 22, 2024
VideoThumbnail
2:53:33

Streamlining Patch Management: AWS Systems Manager's Comprehensive Solution for Multi-Account and Multi-Region Patching Operations

Nov 22, 2024
VideoThumbnail
1:01:07

Accelerate ML Model Delivery: Implementing End-to-End MLOps Solutions with Amazon SageMaker

Nov 22, 2024
VideoThumbnail
15:58

Revolutionizing Business Intelligence: Generative AI Features in Amazon QuickSight

Nov 22, 2024