Protecting backup archives with WORM and Tape Retention Lock

Many AWS customers use AWS Storage Gateway’s Tape Gateway to back up and archive long-term mission-critical on-premises data in Amazon S3 Glacier and Amazon S3 Glacier Deep Archive. Customers in regulated industries are mandated by governance policies or regulatory compliance rules to retain their data for many years, or even indefinitely. For example, large banks, broker-dealers, and securities clearing houses in the financial services industry must adhere to SEC Rule 17a-4(f), which specifies that “electronic records must be preserved exclusively in a non-rewriteable and non-erasable format.”

The need to provide WORM (write once, read many) capabilities to long-term data retention also applies to other industries. For example, consider healthcare, with the Health Insurance Portability and Accountability Act (HIPAA), or financial services, with the Payment Card Industry Data Security Standard (PCI DSS). Besides regulatory compliance, you can use WORM to protect your backup and archives from getting overwritten.

Storage Gateway is HIPAA eligible and compliant with PCI DSS. Tape Gateway now supports WORM and Tape Retention Lock, providing you with data protection features for virtual tapes to address your compliance and security requirements. Using WORM and Tape Retention Lock together enable customers to protect their data from malicious or accidental deletion.

In this blog, I review Tape Gateway’s WORM and Tape Retention Lock capabilities, and walk you through configuring Tape Gateway to take advantage of these new capabilities.

Overview

Tape Gateway serves as a virtual tape library (VTL) and supports leading backup applications. Tape Gateway presents an iSCSI interface and emulates a magnetic tape library that can be integrated into your existing backup or archive workflows.

When you back up your data to AWS via the Tape Gateway, the virtual tapes are stored in a virtual tape library, presented by the gateway. The tape library is backed by Amazon S3, a highly scalable, durable object storage service. The active virtual tapes, which are tapes the backup application is using and writing data to, are stored in the Amazon S3 Standard storage class.

When you no longer need frequent or immediate access to the virtual tapes, you can move the virtual tapes from VTL to archives in Amazon S3 Glacier or Amazon S3 Glacier Deep Archive. All you have to do is eject or export the virtual tapes from their backup application. Tape Gateway stores backups and archives in service-managed Amazon S3 buckets making it easy for you to get started with using AWS Cloud storage. Tape Gateway supports storing your data in Amazon S3, S3 Glacier, and S3 Glacier Deep Archive storage classes.

What is a WORM virtual tape?

With Tape Gateway WORM, you can ensure that data backed up to WORM virtual tapes in your VTL cannot be overwritten or erased. You can only append new data to an active WORM virtual tape. This new capability complements already supported WORM capability of virtual tapes archived in Amazon S3 Glacier or Amazon S3 Glacier Deep Archive, providing you with comprehensive data protection for both active and archived virtual tapes.

What is Tape Retention Lock?

The Tape Retention Lock feature enables you to specify the retention mode and retention period on tapes in Amazon S3 Glacier or S3 Glacier Deep Archive. This prevents tapes from being deleted for a fixed amount of time or up to 100 years. Tape Retention Lock helps you comply with industry regulations on data that you must retain for compliance purposes. Additionally, Tape Retention Lock enables Tape Gateway to be used in environments that are subject to SEC 17a-4, CTCC, and FINRA regulations, which also require WORM storage.

Tape Retention Lock protection works for virtual tapes that are archived and can be configured at the pool (a logical collection of tapes) level using the AWS Management Console, API, AWS CLI, or AWS SDK. Tape Retention Lock period begins when a virtual tape is archived, by ejecting or exporting it from the backup application, and once the virtual tape is in Archived status in AWS. Tape Retention Lock also applies to virtual tapes that have been retrieved from Amazon S3 Glacier or S3 Glacier Deep Archive to S3, and are in Retrieved status.

Tape Retention Lock provides two modes for managing virtual tape retention:

• Governance mode: When using Governance mode, AWS users or groups with specific IAM permissions are able to remove Tape Retention Lock from virtual tapes.
• Compliance mode: This mode is used when you require stronger immutability to comply with industry regulations. In this mode, Tape Retention Lock cannot be removed by any user, including the root AWS user.

Use the Governance mode if you want to protect virtual tapes from being deleted by most users during a pre-defined retention period. Governance mode still allows for some users with special permissions to have the flexibility to alter the retention settings or delete the tapes. Users with the storagegateway:BypassGovernanceRetention IAM permission can override or remove governance mode retention settings. Use Governance mode if you don’t have compliant storage needs.

Use the Compliance mode if there is a requirement to store compliant data. Specifically, the Compliance mode should be used when you never want any user, including the root user in your AWS account, to be able to delete the virtual tapes during the pre-defined retention period. When Compliance mode is activated, the only way to delete a virtual tape under the Compliance mode before its retention date expires is to delete the associated AWS account.

Getting Started using WORM

To get started using Tape Gateway WORM, go to the AWS Storage Gateway console, and select your Tape Gateway. New tapes can be created as either Standard or WORM for the Tape type. The WORM configuration can only be set at the time tapes are created.

Once a WORM tape is created, you see the Tape type listed in the Tape Details tab.

Similarly, you can configure tape auto-create to automatically create a minimum number of WORM tapes according to your business needs. For example, you can use the four-character Barcode prefix to identify workloads that require WORM, such as in the case of HR, Legal, or finance-related backups. You can set the Tape type to Standard for all other non-regulatory workloads.

Once the tapes are automatically created, you can easily identify which tapes are WORM or Standard based on the Barcode prefix.

Once WORM tapes are created, they cannot be deleted from the VTL unless you have the appropriate IAM storagegateway:DeleteTape permission.

Tape Gateway already supports WORM capability on the virtual tapes that are archived in AWS. Virtual tapes archived in Amazon S3 Glacier or S3 Glacier Deep Archive are made read-only tapes when the virtual tapes are retrieved into Amazon S3 for restore purposes, so the virtual tapes cannot be overwritten. You can only read data from virtual tapes once they are ejected or exported by the backup application.

Getting Started using Tape Retention Lock

To begin using Tape Retention Lock, go to the Pools page from the navigation pane on the left in the Storage Gateway console. A pool is a logical collection of tapes that you want to group together to apply the same pool properties. This includes properties such as storage class, retention lock type, and retention period. In addition to the default Amazon S3 Glacier and S3 Glacier Deep Archive pools, you can now create custom pools and enable Tape Retention Lock. All you have to do is select Compliance or Governance from the Retention lock type selection, then specify the number of days for the Retention period. Any tapes created manually, or using tape auto-create configuration and assigned to the custom pool, inherit the retention lock type and retention period for that pool.

You are able to view which pools have Retention lock type set from the Pools page:

You can select a custom pool with your desired Retention lock type when you configure automatic tape creation.

On a pool level, Tape Retention Lock cannot be removed or modified. At a tape level, the only way you can place a tape in a different pool with different retention lock settings is if the retention lock type is set to Governance. You also need the storagegateway:BypassGovernanceRetention IAM permission, which allows you to override or remove governance lock type retention settings. Alternatively, you can wait until the retention period has expired for the Tape Retention Lock configuration to change the Pool a tape is associated with.

Summary

In this blog post, we discussed what Tape Gateway’s WORM and Tape Retention Lock are and how to begin using these features. Specifically, WORM refers to write once, read many capabilities applied to long term data retention, and Tape Retention Lock enables you to specify the retention mode and retention period on virtual tapes in Amazon S3 Glacier or S3 Glacier Deep Archive.

Together, WORM and Tape Retention Lock help customers in regulated industries, such as financial services and healthcare, protect their backup archives in AWS while meeting legal and regulatory requirements. You can be assured your WORM virtual tapes do not get overwritten, and your back up archives will be protected from malicious or accidental deletion with Tape Retention Lock.

To learn more about Tape Gateway, check out the following links:

• Blog: Escaping the pain of physical tape management with AWS Storage Gateway
• Blog: How to easily replace physical tape-based backups with Tape Gateway
• Webinar: AWS Storage Gateway – Escape the pain of physical tape management
• Getting started with Tape Gateway

Thank you for reading about Protecting Backup Archives with WORM and Tape Retention Lock. Please leave a comment in the comments section if you have any questions.