Protect Data on Amazon S3 Against Accidental Deletion or Application Bugs Using S3 Versioning, S3 Object Lock, and S3 Replication

TUTORIAL

Overview

Amazon S3 is an object storage service with industry-leading scalability, data availability, security, performance, and 99.999999999% (11 9s) of data durability. However, even the most durable storage cannot protect against human error or software bugs that could cause your applications to become corrupted or to accidentally delete data. With ransomware events on the rise, customers of all sizes are evaluating additional protection options against malicious tampering with their critical data in Amazon S3.

As with any data, it is best practice to have a backup and to put safeguards in place against malicious or accidental deletion. This Amazon S3 getting started guide shows you how to follow best practices with Amazon S3 Versioning, which allows you to preserve, retrieve, and restore every version of every object stored in an Amazon S3 bucket. Amazon S3 Object Lock can then be added on top of S3 Versioning to prevent data from being deleted or overwritten for a fixed amount of time, or indefinitely. For creating additional copies of your data in another AWS Region for multi-Region protection, Amazon S3 Replication works with both S3 Versioning and S3 Object Lock to automatically copy objects across AWS Regions and separate AWS accounts. Finally, you can bring visibility of your current data protection levels and the usage of these features all together into a single dashboard with Amazon S3 Storage Lens.

What you will accomplish

  • Create an Amazon S3 bucket
  • Enable S3 Versioning to prevent objects from being overwritten
  • Configure S3 Object Lock to prevent accidental deletion of objects
  • Review data protection status of S3 buckets using S3 Storage Lens

Prerequisites

To complete this tutorial, you need an AWS account. Access this support page for more information on how to create and activate a new AWS account.

Note: Since we are demonstrating S3 Object Lock, the data in this lab cannot be deleted until 1 day after it is created.

 AWS experience

Beginner

 Time to complete

30 minutes

 Cost to complete (with free metrics)

Less than $1 (Amazon S3 pricing page)

 Requires

AWS account

 Services used

 Last updated

October 3, 2022

Implementation

Step 1: Create an Amazon S3 bucket

1.1 – Sign in to the Amazon S3 console

  • If you have not already done so, create an AWS account.
  • Log in to the AWS Management Console using your account information.
  • From the AWS console services search bar, enter S3. Under the services search results section, select S3.

Click the screenshots throughout this tutorial to enlarge the image.

1.2 – Create an S3 bucket

  • In the navigation pane on the left, choose Buckets, and then choose Create bucket in the Buckets section.

1.3

  • Enter a descriptive name for your bucket. Bucket names are globally unique across all AWS accounts. If you encounter an error with the name you selected, try another combination. Then, select which AWS Region you would like your bucket created in.

1.4

  • Next, in the Object Ownership section, leave the default setting with ACLs disabled. We recommend that you disable ACLs (access control lists) except in unusual circumstances where you need to control access for each object individually. With Object Ownership, you can disable ACLs and rely on policies for access control. For more information, visit the Amazon S3 User Guide.

 1.5

  • In the Block Public Access settings for this bucket section, the default block public access setting is appropriate for this workload, so leave the default settings.

1.6 – Enable versioning

  • Next, in the Bucket Versioning section, make sure you Enable Bucket Versioning. Versioning in Amazon S3 is a means of keeping multiple variants of an object in the same bucket. You can use the S3 Versioning feature to preserve, retrieve, and restore every version of every object stored in your buckets. With S3 Versioning, you can recover more easily from both unintended user actions and application failures. After versioning is enabled for a bucket, if Amazon S3 receives multiple write requests for the same object simultaneously, it stores all of those objects.

1.7 – Enable default encryption

  • Now, in the Default encryption section, enable default encryption for the bucket. These settings will apply to any objects uploaded to the bucket where you have not defined at-rest encryption details during the upload process. For this workload, under Server-side encryption, select Enable. Then, under Encryption key type, select Amazon S3-managed keys (SSE-S3). If your workload requirements are not satisfied by SSE-S3, you can also use AWS Key Management Service (AWS KMS). For more information about how Amazon S3 uses AWS KMS, see the AWS Key Management Service Developer Guide.

1.8

  • Next, in the Advanced settings section, under Object Lock, select Enable.
  • You can use S3 Object Lock to store objects using a write-once-read-many (WORM) model. S3 Object Lock can help prevent objects from being deleted or overwritten for a fixed amount of time, or indefinitely. You can use S3 Object Lock to meet regulatory requirements that require WORM storage, or add an extra layer of protection against object changes and deletion.
  • Note that this step simply enabled Object Lock on the bucket. We will set specific settings, such as Retention Mode and Retention Period, later in the tutorial.
  • Create the S3 bucket by choosing Create bucket.

Step 2: Set up S3 Versioning

2.1 – Upload an object

  • On your workstation, create a text file that contains the words Version 1 and save it to your workstation.

2.2

  • From the Amazon S3 console, search for the bucket that you created in Step 1, and select the bucket name.
  • Next, select the Objects tab. Then, from within the Objects section, choose Upload.

2.3

  • Then, in the Upload section, choose Add files. Navigate to your local file system to locate the test file that you created above. Select the appropriate file, and then choose Open. Your file will be listed in the Files and folders section. Leave the rest of the options on the default settings, and choose the Upload button.

2.4

  • After the file upload operations have completed, you will be presented with a status message indicating if the upload was successful or not. In this case, the file has uploaded successfully. Then, choose Close.
  • You can now see the object in the S3 console.

2.5

  • On your workstation, edit the file that you created. Change the text to Version 2, then save it with the same file name. Upload the updated file to Amazon S3 by repeating Steps 2.2 through 2.4.

2.6

  • To see a list of the versions of the objects in the bucket, choose the Show versions selection. For each object version, the console shows a unique version ID, the date and time the object version was created, and other properties. 

Now that you have enabled S3 Object Versioning, all versions of that object will continue to be preserved in your Amazon S3 bucket and can be retrieved or restored. With S3 Versioning enabled, only the owner of an Amazon S3 bucket can permanently delete a version. When a DELETE operation is performed on an object, subsequent simple (unversioned) requests will no longer retrieve the object, but it still remains in S3 as a previous version.

When versioning is enabled, a simple DELETE cannot permanently delete an object. Instead, Amazon S3 inserts a delete marker in the bucket, and that marker becomes the current version of the object with a new ID. 

Keeping noncurrent versions of objects can increase your storage costs. You can use lifecycle rules to manage the lifetime and the cost of storing multiple versions of your objects. You can have lifecycle rules configured to permanently remove these older versions or for additional windows of protection at a lower storage cost. You can also set up a lifecycle rule that archives all of your previous versions to the lower-cost Amazon S3 Glacier Instant Retrieval or Amazon S3 Glacier Flexible Retrieval storage classes and then deletes them after 100 days, giving you a 100-day window to roll back any changes on your data while lowering your storage costs.

Select the check box to the left of the top-level object, and choose Open. This object represents the most recent version of the file, and should say Version 2. Repeat the process on the older version of the object and confirm that it says Version 1.

2.7 – Demonstrate deleting objects

  • Turn off Show versions to collapse the list of versions. 

Enter delete in the text input field and choose the Delete objects button.

  • Choose Close.
  • Then, on the Objects tab, you will see the object appears to have been deleted.
  • Turn on Show versions. Notice that the top-level object is now listed as a Delete marker under the Type column, but both versions of the object are still available.

2.8 – Demonstrate restoring an object

  • To restore the object, delete the Delete marker by selecting the check box of the top-level object. Then, choose Delete.
  • Enter permanently delete in the text input field and choose Delete objects.
  • Choose Close.
  • Deleting the Delete marker has restored the object.
  • Select the check box to the left of the top-level object, and choose Open. This object should open as Version 2.
  • Select the check box to the left of the older version of the object, and choose Open. This object should open as Version 1.

2.9 – Permanently delete a specific version of an object

We can also delete a specific version of an object without creating a Delete marker. In the next step, we will delete the most recent version of the object.

  • With the Show versions toggle turned on, select the check box to the left of the top-level object, and choose Delete.
  • Enter permanently delete in the text input field and choose Delete objects.
  • We have now deleted the most recent version of the object.
  • To confirm this, select the check box to the left of the remaining object, and choose Open
  • Making the above selection will then open a new tab in your browser, with the text Version 1.

Step 3: Set up S3 Object Lock

3.1

  • Show the detailed features at the bucket level. From your bucket, choose Properties.

3.2

  • Navigate down to the Object Lock section. Then, choose Edit.

3.3

  • In the Default retention section, choose Enable.
  • Set the Default retention mode to Compliance.
  • For the Default retention period, enter 1 in the text input field and leave the dropdown box as Days. Then, choose Save changes.

We have now set the default Object Lock settings for any new objects that are uploaded to the bucket. Existing objects in the bucket are not impacted by these settings. To lock existing objects, you can use S3 Batch Operations.

In governance mode, you can't overwrite or delete an object version or alter its lock settings unless you have the s3:BypassGovernanceRetention permission. The S3 console by default includes the x-amz-bypass-governance-retention:true header. If you try to delete objects protected by governance mode and have s3:BypassGovernanceRetention permissions, the operation will succeed. For this reason, we are using compliance mode in this tutorial.

When a lock is placed on an object using Compliance mode, the object version cannot be deleted by any user, including the root user, until the retention period expires. In addition, its retention mode can't be changed, and its retention period can't be shortened.  

This tutorial will demonstrate how even a user with administrative rights is unable to delete objects protected with compliance mode. Note that objects that are locked and cannot be deleted will continue to be billed. To minimize costs for this tutorial, ensure you have set the default retention to only 1 day, and only upload small files.

3.4 – Upload a new object

  • From within the Objects tab, choose Upload

3.5

  • Then, in the Upload section, under Files and folders, choose Add files. Navigate to your local file system and select any small file to be used for testing, and then choose Open. Your file will be listed in the Files and folders section. Leave the rest of the options on the default settings, and choose Upload.

3.6

  • After the file upload operations have completed, you will be presented with a status message indicating if the upload was successful or not. In this case, the file has uploaded successfully. Then, choose Close.

3.7

  • From the Objects tab, choose the new test object that you uploaded.

On the object Properties tab, review the Object management overview section. Note that the Object Lock defaults that we set have been applied to the object. You can also override these defaults when uploading a new object, and extend (but not reduce) the retention date applied to a locked object.

3.8 – Attempt to delete the object

In this section, we will explore another method of permanently deleting a specific version of an object, without creating a Delete marker. 

  • Choose the Versions tab.  
  • Select the check box to the left of the object, then choose Delete.

Enter permanently delete in the text input field and choose Delete objects.

  • You should receive an error message that the object cannot be deleted. This is the expected behavior when using Object Lock in compliance mode. You will need to wait until the retention period (which we set to 1 day) has expired before this object can be deleted.
  • Then, choose Close to exit this window.

Step 4: Explore additional S3 data protection features

Although out of scope for this tutorial, there are two other S3 data protection topics that you should be aware of:

S3 Replication enables automatic, asynchronous copying of objects across Amazon S3 buckets. Buckets that are configured for object replication can be owned by the same AWS account or by different accounts. You can replicate objects to a single destination bucket or to multiple destination buckets. The destination buckets can be in different AWS Regions or within the same Region as the source bucket.

S3 Replication requires S3 Versioning to be enabled on both the source and destination buckets.  

If the source bucket has S3 Object Lock enabled, the destination bucket(s) must also have S3 Object Lock enabled. To enable replication on a bucket that has Object Lock enabled, you must first contact AWS Support. When Amazon S3 replicates objects that have retention information applied, it applies those same retention controls to your replicas, overriding the default retention period configured on your destination buckets. If you don't have retention controls applied to the objects in your source bucket, and you replicate into destination buckets that have a default retention period set, the destination bucket's default retention period is applied to your object replicas. 

Logging options

Amazon S3 is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in Amazon S3. CloudTrail captures a subset of API calls for Amazon S3 as events, including calls from the Amazon S3 console and code calls to the Amazon S3 APIs.

Server access logging provides detailed records for the requests that are made to a bucket. Server access logs are useful for many applications. For example, access log information can be useful in security and access audits. It can also help you learn about your customer base and understand your Amazon S3 bill.

Use AWS Backup to centralize and automate data protection across AWS services and hybrid workloads. AWS Backup offers a cost-effective, fully managed, policy-based service that further simplifies data protection at scale. AWS Backup also helps you support your regulatory compliance or business policies for data protection.  

Amazon S3 is one of the many supported AWS Backup resources. AWS Backup also leverages S3 Versioning.

Step 5: View S3 data protection metrics with Amazon S3 Storage Lens

Amazon S3 Storage Lens delivers organization-wide visibility into the data protection status of your data stored in Amazon S3. S3 Storage Lens can easily show you the percentages of your data that are encrypted, versioned, replicated, and object locked. This data protection visibility in S3 Storage Lens can be viewed for your data across multiple Amazon S3 buckets, AWS accounts, and AWS Organizations at no additional charge with 14 days of historical data protection levels to immediately get started.

You can use S3 Storage Lens to see a summary of data protection metrics, expressed as a percentage of the total amount of data. These include: current version bytes, noncurrent version bytes, encrypted bytes, replicated bytes, and object lock bytes.

5.1

  • From the S3 console, find Storage Lens in the navigation pane.
  • Choose Dashboards, then default-account-dashboard.

5.2

  • Select the Data protection tab to see statistics relating to versioning, encryption, replication, and object lock.

Step 6: Clean up

In the following steps, you clean up the resources you created in this tutorial. It is a best practice to delete resources that you are no longer using so that you do not incur unintended charges.

Note that you will need to wait 1 day for S3 Object Lock to expire before attempting these steps.

6.1 — Delete test objects  

  • If you have logged out of your AWS Management Console session, log back in. Navigate to the S3 console and select Buckets in the navigation pane. First you will need to delete the test object(s) from your test bucket. Select the radio button to the left of the bucket you created for this tutorial, and then choose Empty.
  • In the Empty bucket page, enter permanently delete into the Permanently delete all objects confirmation box. Then, choose Empty to continue.
  • Next, you will be presented with a banner indicating if the deletion has been successful. Choose Exit.

6.2 – Delete test bucket

  • Finally, you need to delete the test bucket you have created. Return to the list of buckets in your account. Select the radio button to the left of the bucket you created for this tutorial, and then choose Delete.
  • Review the warning message. If you desire to continue deletion of this bucket, enter the bucket name into the text input field confirmation box and choose Delete bucket.

Conclusion

Congratulations! You have learned how to protect data in Amazon S3 using S3 Versioning and S3 Object Lock, and reviewed these settings using S3 Storage Lens.

Was this page helpful?

Next steps

To learn more about Amazon S3, visit the following resources.