Serverless Fixity for Digital Preservation Compliance

Amazon S3 Standard, S3 Standard–IA, S3 One Zone-IA, S3 Intelligent-Tiering, S3 Glacier, and S3 Glacier Deep Archive storage classes are all designed to provide 99.999999999% durability of objects over a given year. These services are designed to sustain concurrent device failures by quickly detecting and repairing any lost redundancy, and they also regularly verify the integrity of data using checksums.

Serverless Fixity for Digital Preservation Compliance makes it easier for customers who require an on-demand fixity check process to validate the checksums for compliance and audit requirements. Using this solution, AWS customers can check the integrity of their objects stored in any Amazon S3 storage class using either the MD5 or SHA1 checksum algorithm without having to incur the cost and complexity of third-party software.

With Serverless Fixity for Digital Preservation Compliance, you can start the fixity check process using the AWS Management Console, Amazon API Gateway, or the AWS Command Line Interface (CLI). Fixity check results are mailed to subscribers using Amazon Simple Notification Service (Amazon SNS) notifications.

Overview

The diagram below presents the architecture you can build using the code example on GitHub.

Serverless Fixity for Digital Preservation Compliance | Architecture Diagram
 Click to enlarge

Serverless Fixity for Digital Preservation Compliance architecture

Serverless Fixity for Digital Preservation Compliance launches an AWS Step Functions state machine, AWS Lambda functions, and Amazon SNS. The AWS Step Functions state machine workflow restores and computes using either an MD5 or SHA1 checksum algorithm, and validates objects stored in your Amazon S3 buckets.

Serverless Fixity for Digital Preservation Compliance orchestrates the fixity check process in various states. If necessary, the workflow restores the object from the Amazon S3 Glacier or Amazon S3 Glacier Deep Archive storage class. Then, the process incrementally computes the fixity. After the MD5 or SHA1 checksum is calculated, the calculated checksum is validated with the original checksum value stored with the object. The results of the fixity check process are sent to an Amazon SNS topic, which is then sent to subscribers.

Serverless Fixity for Digital Preservation Compliance also creates an Amazon API Gateway endpoint that provides a RESTful API to start and monitor the fixity check process. The RESTful API requires authentication using valid AWS Identity and Access Management (IAM) credentials. By default, Serverless Fixity for Digital Preservation Compliance works with Amazon S3 buckets and objects in your existing AWS account.

Serverless Fixity for Digital Preservation Compliance

Version 1.1.0
Last updated: 09/2021
Author: AWS

Additional resources

Did this Guidance help you?
Provide feedback 

Features

Fixity checks using native AWS services

Perform fixity checking on objects stored in any Amazon S3 storage class using native AWS services.

Meet compliance requirements

Meet compliance requirements that require a periodic re-computation and comparison of the checksum against external records of that checksum.

Flexible fixity check options

Start the fixity checks using the AWS Management Console, Amazon API Gateway, or the CLI.

Receive notifications for fixity check results

Receive Amazon SNS notifications with the results of the fixity checks.
Build icon
Deploy a Solution yourself

Browse our library of AWS Solutions Implementations to get answers to common architectural problems.

Learn more 
Find an APN partner
Find an APN Partner

Find AWS certified consulting and technology partners to help you get started.

Learn more 
Explore icon
Explore Solutions Consulting Offers

Browse our portfolio of Consulting Offers to get AWS-vetted help with solution deployment.

Learn more