TeraMach Helps Library and Archives Canada Simplify Access to Public Information

Library and Archives Canada partnered with TeraMach, an AWS Public Sector Partner, to implement a digital preservation solution hosted in the AWS Cloud. The collaboration resulted in simpler data management, improved long-term preservation, and increased access to the country’s archives for all Canadians. The system runs on Amazon EC2, using Amazon S3 for storage and Amazon VPC for internal networking.

The collection managed by Library and Archives Canada (LAC), a government department tasked with preserving Canada’s documentary heritage, is truly vast. It includes 20 million publications in various languages, 250 linear kilometers of government and private text records, 30 million photographic images, 550,000 hours of audio and video recordings, and much more. Its digital holdings total 18 petabytes (PB), growing by 1.25 PB annually—a figure set to increase rapidly in the coming years. With the growing capabilities of the cloud, LAC saw an opportunity to improve the scalability, agility, and efficiency of its operations while improving access to information for the people of Canada.

“Several years ago, we announced that by 2017, anything created by government departments had to be submitted in digital format,” says Sylvain Bélanger, Director General at LAC. “We needed to be ready to ingest and preserve that data for decades and centuries to come. That led us to develop an overall digital strategy for LAC, including the need for a cloud-based digital preservation system.” In keeping with the cloud-first mandate of the Canadian government, the solution was ultimately built in the AWS Cloud.

LAC partnered with TeraMach, an AWS Advanced Consulting Partner and an AWS Public Sector Partner with a Competency in Government. AWS provided the scalability and security the agency required as well as three availability zones in the Canada Central region. Supported by in-house technical experts, TeraMach turned to its experience with AWS to successfully deliver the Digital Asset Management System.

The solution is built around a managed application that ingests files, performs automated workflows based on metadata, and deposits them in an encrypted Amazon Simple Storage Service (Amazon S3) bucket. From there, it can be ingested and archived by Preservica, a digital preservation technology. The entire system is hosted on AWS, using Amazon Elastic Compute Cloud (Amazon EC2) for compute and Amazon Virtual Private Cloud (Amazon VPC) for networking. The organization is exploring the use of low-cost Amazon S3 storage tiers for long-term archiving.

By partnering with TeraMach, LAC was able to bring a standards-based, secure, and compliant solution to production faster than if it had tried to develop it internally. TeraMach’s experience with Government of Canada security standards and AWS security controls enabled LAC to obtain an Authority to Operate (ATO) certification quickly. This exceeded the expectations of LAC and ultimately contributes to the preservation of Canadian heritage for generations to come.

The average person puts files on digital media without considering long-term consequences, but LAC is tasked with ensuring the accessibility of those materials for centuries to come. This complex task includes monitoring digital collections for obsolescence and decay, migrating files at risk of digital loss, and managing information to ensure it remains searchable and available. “By making digital copies easily accessible through the AWS Cloud, people won’t have to physically travel to the archives to use the collection, but can access it over the web,” says Roslynn Ross, Director of Digital Preservation and Migration at LAC.

The cloud-based solution has streamlined the preservation workflow, freeing specialized staff for value-added tasks. For example, the organization previously had multiple ways for publishers to submit books and periodicals, including via email, requiring significant manual work from LAC librarians. Now, the web front-end handles the ingestion process through a standardized form. “We can control the quality of the data early in the process,” says Bélanger. “This reduces the chance of human error and allows librarians and archivists to focus on reaching new audiences and addressing the backlog of digital collections.”

TeraMach has worked with LAC to ensure the solution met not only today’s requirements but was also suitable for long-term strategic needs. “The team has been hand in hand with us to define our strategy, design the technical solution, and implement it effectively,” says Bélanger. “TeraMach helped us expand our understanding of what a solution could be. We met with the team weekly and worked together to adapt as new requirements became apparent.”

For its part, TeraMach is proud to partner on a project with such a significant impact on Canadians. “This has been a collaboration from the beginning,” says Claudio DiGirolamo, Executive Vice President, Managed Service Sales at TeraMach. “It’s been really exciting to be here from the start and see how it has evolved.”

LAC is continuing to roll out new functionality, such as the ability for universities to submit research theses and automatically link them to the LAC catalog. In addition, the organization is piloting digital ingestion with government departments in preparation for the full-scale implementation of its digital-only mandate. “This system puts us at the forefront of digital ingestion and preservation, so we can focus on helping Canadians across the nation access and use the collection,” says Bélanger.