The Rock & Roll Hall of Fame is preserving rock music history with Amazon S3 Glacier Deep Archive

The Rock & Roll Hall of Fame – aka Rock Hall – in Cleveland, Ohio is on a mission to engage, teach and inspire through the power of ROCK! The Rock Hall contains thousands of items of memorabilia from every era and genre of rock and roll, and millions of documents, audio recordings, photographs and video files.

The Rock Hall wanted to make this treasure trove more widely available to fans and scholars, so it established a Library & Archives Department, which includes an on-site digitization and conservation lab.

The department’s main aim is to collect, preserve and provide access to the Rock Hall’s archival material, including master files of induction ceremonies, B roll, oral histories, artist interviews, symposiums on music and original concerts, in a centralized digital media repository.

When the Rock Hall was established in 1995, LTO tape storage was a good solution for managing large files. But in the intervening years, the LTO infrastructure has become expensive to maintain and prone to failure. Additionally, hardware and software failures, as well as on-site storage limitations, made the LTO tapes increasingly hard to access.

“As the tech landscape progressed from the non-digital to digital, our archival storage system had become unmanageable and unsustainable,” says Tim Parnin, VP of digital technology at the Rock Hall. “We had unique and priceless files on media that we couldn’t access anymore. We wanted to get our digital files sitting in a singular location that could be searchable and discoverable by our internal team, and shared with the public.”

In 2018, Rock Hall leaders evaluated the organization’s technical infrastructure and decided to adopt Amazon Simple Storage Service (Amazon S3) and, subsequently, Amazon S3 Glacier Deep Archive storage to preserve the museum’s digital media.

“We needed something that could handle the sheer volume of our assets, as well as a rugged, expandable solution we weren’t going to outgrow, and one we could race into the future with,” says Parnin.

Using Amazon S3 and S3 Glacier Deep Archive gave the Rock Hall confidence that its digital media would be preserved and easily accessible at an affordable price.

Cold as ice

The Rock Hall still needed to recover the data on the LTO tapes. Working with AWS, the project team ingested the files into S3 Glacier Deep Archive via six AWS Snowball Edge Storage Optimized devices. Using AWS Snowball Edge helped to address common challenges with large-scale data transfers including high network costs, long transfer times and security concerns.

Tape Ark and its strategic partner Seagate are experts in tape-to-cloud migration and were brought in to help. “Tape-to-cloud projects are often complicated, not only by the need to read old tapes, but by the old formats that the tapes were originally written in,” says Tape Ark CEO and founder, Guy Holmes. “Tape Ark has designed a range of data format converters that allows all major back-up formats to be restored without the need to use the original software that created them.”

With Tape Ark’s help, the Rock Hall restored all the data from the LTO tapes. “We ordered AWS Snowball Edge devices and shipped them straight to Tape Ark, who were able to extract the data from the LTO tapes for us,” says Alex DeCamillo, software developer at the Rock Hall. “Tape Ark then sent the Snowball devices loaded with data back to AWS and our data went right into our Amazon S3 bucket.”

The Seagate Powered by Tape Ark team achieved 0% data loss, recovering all located 2000 preservation files and locating an additional 109 preservation-level videos. Once the digital media was in Amazon S3 in the AWS Cloud, Amazon S3 life cycle policies were set up to automatically move the files from Amazon S3 into S3 Glacier Deep Archive to lower storage costs.

Don’t stop me now

The Rock Hall has now recovered and preserved more than 2000 files and 280TB of data to ensure future generations have access to rock and roll history. Amazon S3 and S3 Glacier Deep Archive have also helped improve efficiency.

“Before, we would have to download a preservation file overnight and hope it worked by the time we came back in the morning,” says Jennie Thomas, director of archives at the Rock & Roll Hall of Fame. “Now we can download that same file in about 15 minutes.”

She continues: “In addition to having all our assets in S3 Glacier Deep Archive, knowing that AWS is making duplicate copies and has disaster planning and recovery taken care of is a huge load off my mind. The fact that we were able to realize all of these benefits and still save significant costs with S3 Glacier Deep Archive compared to the cost and overhead of managing our own tape library was an added bonus.”

Gimme shelter

With the Rock Hall’s assets safe, the next step was to allow employees to easily access them in their daily work. “The files we created over the years didn’t have embedded metadata,” says Thomas. “If you didn’t have institutional memory to know what the file name meant and what collection it tied to in the archives, it would be difficult to find the assets you needed.”

Rock Hall partnered with digital asset migration specialist, GrayMeta. Using GrayMeta’s AI-enabled platform, including facial recognition, unlocked a wealth of metadata in the archive, allowing the Rock Hall team to understand their assets and instantly search through a load of images, audio and video.

With the project complete, the Rock Hall can now digitize, preserve and enjoy easy access to a mass of content. “We now have the reassurance we needed to know our archives are protected and future generations from around the world can enjoy the history of rock and roll,” concludes Parnin.

This article first appeared in the July 2020 issue of FEED magazine.