Customer Stories / Media & Entertainment / United Kingdom

2024
BBC Logo

The BBC Preserves 100 Years of History Using Amazon S3

Learn how the BBC, a UK public service broadcaster, safely migrated its flagship archive to Amazon S3 Glacier Instant Retrieval.

Preserved

BBC Archive Content for future generations

25 PB

migrated in 10 months (120 TB/day); retired physical infrastructure

Reduced

overall infrastructure costs using Amazon S3 Glacier Instant Retrieval and S3 Intelligent-Tiering

Optimized

data availability and enhanced data and content accessibility

Data lake

foundation with standardized storage for future enrichment opportunities

Overview

The British Broadcasting Corporation (BBC) Archives Technology and Services team needed a modern solution to centralize, digitize, and migrate its 100-year-old flagship archives. The team wanted to merge its archives to enhance the preservability and accessibility of the media for future use.

Because the BBC had experience using Amazon Web Services (AWS), it began using Amazon Simple Storage Service (Amazon S3) Glacier Instant Retrieval, which is an archive storage class that delivers the lowest-cost storage for long-lived data that is rarely accessed and requires retrieval in milliseconds. By migrating its archives to AWS, the BBC optimized data accessibility, created cost efficiencies, freed up space used by physical infrastructure, and supported a transition into the future of archive preservation.

Broadcast Booth

Opportunity | Using AWS to Preserve Data and Improve Data Accessibility for the BBC

BBC Archives Technology and Services is the custodian of the BBC’s 100-year-old archives, with 16 million assets collected throughout the broadcaster’s existence—from historical film to modern digital media. The team focuses on three goals: unifying all of the archives’ delivery content, making the archives more accessible through digitization, and preserving the media with modern solutions. “We want to have a forward-looking strategy, with tools like flexible storage and compute that facilitate the use of machine learning,” says Brendan Mallon, head of product and services in BBC Archives Technology and Services. “Our goal is to safeguard the content in the archives so that it’s accessible for another 100 years.”

The broadcaster’s data had been split into separate genre repositories for news, sports, radio, and programs. It wanted to standardize its supply chain and workflows to develop a sustainable and centralized plan for archiving. It had been successfully using on-premises infrastructure, but in 2017 it saw a need to reduce the complexity of its systems. Content aggregation was unnecessarily onerous with the existing, disparate datasets. So, the team embarked on a 5-year mission to consolidate its various storage application layers and become more sustainable.

Because the BBC had been running its media asset processing system on AWS for years, it was a practical next step to migrate to an AWS solution that could contribute to the longer-term preservation strategy. “We wanted a consistent approach to extract value from inconsistent datasets, create an authoritative single catalogue that matches the media, and drive value for our audiences,” says Mallon. “Using AWS, we can standardize storage for all of our content.”

kr_quotemark

By using Amazon S3 Glacier Instant Retrieval, we can benefit from expedited retrieval from the archive while having cost flexibility.”

Mark Glanville
Senior Technical Architect, BBC Archives Technology and Services

Solution | Migrating 120 TB per Day and Reducing Costs with Optimized Storage

Due to the scale of the archive data, a network migration was the most practical option to automate. To complete the content migration, the BBC used its existing infrastructure powered by AWS Direct Connect, which is a cloud service that helps users create a dedicated network connection to AWS for smooth and reliable data transfers at a massive scale for near real-time analysis, rapid data backup, or broadcast media processing. In November 2022, after about 12 months of planning and consulting with Cloudfirst.io (Cloudfirst), an AWS Partner, the migration kicked off.

At peak, the team migrated 120 TB of data per day, using AWS Direct Connect to transfer sizable amounts of content to AWS. Within 10 months, the team had transferred 25 PB of data out to the cloud. By doing so, it could retire one of its legacy tape-based media repositories and develop a next-generation abstraction between media asset management systems and public cloud storage. “We were able to retire half the archive’s physical infrastructure,” says Mark Glanville, senior technical architect in BBC Archives Technology and Services. “This frees up a huge amount of technical space and power in some precious real estate in Central London.”

The BBC migrated most of its data to Amazon S3 Glacier Instant Retrieval. The solution provided the ideal type of flexible storage for the sorts of data that were stored in the BBC’s archives. Because the team had used AWS for its media asset processing system, which housed about 3 PB of media, it had experience with Amazon S3 solutions. “We worked alongside the AWS team to select the right storage class for the bulk of our content,” says Glanville. “By using Amazon S3 Glacier Instant Retrieval, we can benefit from expedited retrieval from the archive while having cost flexibility.”

The team uses a combination of Amazon S3 Glacier Instant Retrieval and Amazon S3 Intelligent-Tiering, a cloud storage class that delivers automatic storage cost savings without performance impact or operational overhead. The BBC can choose between these two storage classes depending on its expected level of access without compromising on performance.

“By using Amazon S3 Glacier Instant Retrieval and Amazon S3 Intelligent-Tiering, we get archive-like pricing models for content that we previously had in relatively hot storage,” says Tom Cartwright, executive product manager at the BBC. “This is really valuable because we can make decisions early on in a project about where our data lives.”

Outcome | Creating a Data Lake for Machine Learning Opportunities

With 100 years of content now in the archives, the BBC is focused on the future of data standardization and preservation. It plans to tackle this challenge with a next-generation data lake that consolidates the physical and digital estates. The BBC looks to improve discovery through machine learning, using tools like speech-to-text and facial recognition. Using machine learning tools greatly enhances the searchability of the archives, making it easier to search for just about anything. The goal is improved preservation, availability, and innovation to continually deliver value to audiences for the next 100 years.

“Our shared vision is to set up the business for a sustainable future,” says Mallon. “We want our content to be as discoverable and as accessible as possible.”

About BBC

The British Broadcasting Corporation (BBC) is a 100-year-old public service broadcaster that serves millions of people in the United Kingdom and around the world. The company has an expansive multimedia portfolio of television channels, radio networks, digital services, and news services.

AWS Services Used

Amazon S3

Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance.

Learn more »

Gain free, hands-on experience with S3 »

Amazon S3 Glacier Instant Retrieval

Amazon S3 Glacier Instant Retrieval is an archive storage class that delivers the lowest-cost storage for long-lived data that is rarely accessed and requires retrieval in milliseconds.

Learn more »

Amazon S3 Intelligent-Tiering

Amazon S3 Intelligent-Tiering is the only cloud storage class that delivers automatic storage cost savings when data access patterns change, without performance impact or operational overhead.

Learn more »

AWS Direct Connect

The AWS Direct Connect cloud service is the shortest path to your AWS resources. While in transit, your network traffic remains on the AWS global network and never touches the public internet.

Learn more »

More Media & Entertainment Customer Stories

no items found 

1

Get Started

Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today.