Amazon S3 Glacier for infinite video storage with Ross

This post is authored by Chris Kelly, Product Manager, Asset Management & Storage, Ross Video. The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.

As a Product Manager for a Video Asset Management System, I’m intimately aware of the challenges that vast volumes of video present. Our typical customers produce 6–8 hours of edited video content a day. Much of it is repetitive or throw-away media like “tease and feed” video. But an hour or two of that media can represent thousands of dollars worth of work every day. Not to mention the priceless value to the historical record for the producer and the communities they serve.

That’s why when it came time to develop an integration with a cloud storage provider, we turned to Amazon Web Services (AWS).

Defining the problem

Our customers archive an hour or two of video material a day. But every customer is unique and has different creative, business, and technical pressures that dictate how they want that material archived. Some want the material more compressed to save space, while others want to keep it in the original broadcast format. Today, in a 720p or 1080i environment, that could be as much as 30 GB of data a day, or 8 TB a year. Move to 1080p, and that jumps to 33 TB a year. 4K UHD? 98 TB per year. And once something is archived, it’s rarely purged. That interview with a young, up-and-coming city council person could be valuable in twenty years when they become a senator or president. The content captured today could be nothing, or it could be everything. Content creators rarely know the gravity of a moment until they turn to the archive and find that the material is gone.

To us, an archive isn’t a product. It’s a workflow, it’s a strategy. Archive isn’t a question of who or what, but rather a question of where, why, and how. Where do you store this valuable media? Why are you saving it? How do you archive in the easiest, most cost and time efficient way possible? An archive grows to infinity, which means cost grows to infinity. This media has no expiration date. On the 10th anniversary of a significant story, our customers need every frame of video possible. Same on the 25th anniversary, or the 50th anniversary. And an archive strategy could include saving the media in multiple locations in the event of a catastrophic loss of data.

Our customers value:

• Speed to air
• Reliability and redundancy
• Ease of use
• Cost efficiency

How can our customers meet those values with their archive strategy? In almost every case, Amazon Simple Storage Service Glacier (Amazon S3 Glacier) is the right option.

The benefits of Amazon S3 Glacier

As the archive grows, so does the impracticality of on-premises physical media. Customers can and should keep material locally as long as possible. Spinning disk arrays can hold years and decades worth of material. Retrieval is not any faster than media that lives inside the walls of the producer. The question that presents itself is when it comes time to put content to removable media like tape. At that moment, the material must be managed by a person. With tape, that means maintaining not only the tape, but the drives used to read those tapes. As tape standards change, the material on old tapes needs to be migrated to the latest and greatest tape format. All the while, the archive continues to grow, and every technology change requires more and more work. This is one major benefit to Amazon S3 Glacier; maintenance is offloaded to AWS. AWS manages the safety and security of that content. Both Amazon Simple Storage Service (Amazon S3) and S3 Glacier provide 99.999999999% durability of objects over a given year, meaning if you store 10,000,000 video files in S3 you can on average expect a loss of a single object once every 10,000 years.  There is no need to move content from one tape to another or one drive array to another. Once the content is in S3 Glacier, it’s there for as long as they want.

Another benefit is speed. Once content has aged out of a spinning disk solution, that content is no longer immediately available. The content must be retrieved from a tape system or from the cloud. Complex tape systems can manage the tape movement from shelves to drives, but those systems are complicated and costly. In many situations, the customer must physically find the tape and insert it into the tape drive. That takes up valuable time in that person’s daily routine. Using S3 Glacier expedited retrieval, AWS delivers the file within 5 minutes and requires no extra work for the customer. It’s slower than immediate access, but it comes with a significant cost benefit, which brings us to the third benefit.

S3 is fantastic value, but costs can still add up given the volume of storage demanded by video. S3 Glacier offers a pragmatic compromise where data can be accessed well within customers’ tolerance, but at a price that is a fraction of S3. At 100 TB, the monthly cost of S3 Glacier is less than a quarter of S3, providing substantial savings. It also results in substantial savings in comparison to the cost of maintaining on-premises storage that requires both attention and periodic replacement.

Lastly, S3 Glacier is something that can be implemented on a moment’s notice. Unlike other physical-based options, customers can archive immediately without limits once they establish their AWS account.

Provisioned capacity

S3 Glacier does require provisioned capacity to offer expedited retrieval, which adds $100/month to storage costs. As the volume grows, however, the less weight that cost carries. In as little as 6 TB of storage, the customer meets their break even point versus S3. In a fast-paced media environment, we don’t see anything other than expedited retrieval having significant value, other than perhaps disaster recovery (DR).

Streamline to keep content safe

At Ross Video, the selection of a partner for long-term infinite storage carried major implications, and AWS proved up to the challenge. In upcoming releases of Streamline, our asset management solution, we will introduce rules-based toolsets to automate the process of moving content between multiple storage options. This is where S3 Glacier shines. The value, reliability, speed, and efficiency further simplify the job of archiving this daily generated content. Decades from now, our customers can have complete trust that the content they rely on is safe and sound, ready for air, to tell their stories.

For more information on Ross Video’s Streamline product, please visit the Ross website.