Find public Amazon S3 buckets in your AWS account

Data is key to business, and securing it from unintended access is a critical business activity. As cloud usage increases, this can be a significant task to address. You want to verify that you aren’t unintentionally exposing or sharing data publicly.

Under the Shared Responsibility Model, AWS is responsible for protecting the infrastructure that runs AWS services like Amazon Simple Storage Service (Amazon S3). Your responsibility includes managing access to your data. By default, S3 buckets are private and can be accessed only by users who are explicitly granted access. Additionally, all newly created S3 buckets by default have Amazon S3 Block Public Access enabled, access control lists (ACLs) disabled, and all new objects encrypted. Although these new defaults create a strong security posture, you are still responsible for monitoring your users who may re-configure these settings on your S3 buckets.

In this post, we walk you through the AWS services that you can use to detect S3 buckets that your users have configured for public access across different AWS Regions in your AWS account and AWS Organizations. First on the list is IAM Access Analyzer for S3, which is available at no additional cost in the S3 console and should be your go-to method to identify and remediate public buckets. If you are interested in other AWS services that offer the ability to identify public buckets, you can choose the service that best fits your requirements based on factors like compliance, service features, and cost.

• IAM Access Analyzer for S3
• AWS Config
• AWS Security Hub
• Amazon GuardDuty
• AWS Trusted Advisor
• Amazon Macie
• AWS CloudTrail

IAM Access Analyzer for S3

You can use IAM Access Analyzer for S3 from the Amazon S3 console to review buckets with bucket ACLs, bucket policies, and access point policies that grant public access. IAM Access Analyzer for S3 alerts you to buckets that are configured to allow access to anyone on the Internet or other AWS accounts. For each public or shared bucket, you receive findings that report the source and the level of public or shared access. When a bucket policy, Access Point policy, or ACL is added or modified, IAM Access Analyzer for S3 generates and updates findings based on the change in about 30 minutes. Findings related to account-level Block public access settings or Multi-Region Access Point configurations may not be generated or updated for up to six hours after you change the settings. The following screenshot illustrates S3 buckets with public access:

To enable IAM Access Analyzer for S3, you must create an analyzer in each Region that you want to monitor access to your Amazon S3 resources. IAM Access Analyzer for S3 only works for analyzer with an account as a zone of trust. The following screenshot illustrates creating an analyzer:

AWS Config

AWS Config is a service that lets you assess, audit, and evaluate the configurations of your AWS resources. AWS Config continuously monitors and records your AWS resource configurations and lets you automate the evaluation of recorded configurations against desired configurations using AWS managed rules and user-defined (custom) rules. You can enable AWS Config recording in AWS accounts and the Regions that you want to monitor using AWS Management Console, AWS Command Line Interface (AWS CLI), and AWS SDK’s.

You can use AWS Config to detect and alert on public S3 buckets by performing the following steps:

1. Set up an S3 bucket to receive AWS Config recording in the desired account and Region.
2. Deploy two AWS Config rules: “s3-bucket-public-read-prohibited” and “s3-bucket-public-write-prohibited.”
3. Create an Amazon SNS Topic: create an Amazon Simple Notification Service (Amazon SNS) topic and one or more email subscriptions for this topic.
4. Create an Amazon EventBridge rule: use the following rule that identifies events related to the two AWS Config rules that were deployed in Step 1.

   {
     "source": ["aws.config"],
     "detail": {
       "requestParameters": {
         "evaluations": {
           "complianceType": ["NON_COMPLIANT"]
         }
       },
       "additionalEventData": {
   "managedRuleIdentifier":["S3_BUCKET_PUBLIC_READ_PROHIBITED","S3_BUCKET_PUBLIC_WRITE_PROHIBITED"]
       }
     }
   }

5. Create an email target for the EventBridge rule: select the SNS topic created in Step 2 as a target for the rule. This allows findings to be sent to the email addresses configured in the SNS topic subscriptions.

You can use an AWS Config aggregator and advanced queries to view the resource configuration and compliance data aggregated from multiple accounts and multiple regions. You can integrate AWS Config with AWS Security Hub to have a comprehensive view of your security state in AWS. Read more about Security Hub in the following section of this post.

AWS CloudTrail

AWS CloudTrail is an AWS service that helps you enable operational and risk auditing, governance, and the compliance of your AWS account. Actions taken by a user, role, or an AWS service are recorded as events in CloudTrail. CloudTrail can help you find public buckets in AWS by tracking the creation and modification of S3 buckets, including changes to bucket policies and ACLs. By enabling CloudTrail logs and analyzing the events related to S3 buckets, you can identify which buckets are publicly accessible and which have improper access controls in place.

You can use CloudTrail to detect and alert on public S3 buckets by performing the following steps:

1. Enable CloudTrail: In your Console, navigate to the CloudTrail service. Then, create a new trail and select the S3 bucket where you want to store the CloudTrail logs.
2. Analyze CloudTrail logs: Once CloudTrail is enabled, it starts logging the events related to your AWS account. To identify public buckets, you can filter the CloudTrail logs via Amazon S3 API operations such as CreateBucket, PutBucketAcl, and PutBucketPolicy.

The following screenshot illustrates a public S3 bucket event in CloudTrail:

The following screenshot illustrates Principal as a wildcard in the S3 bucket policy:

3. Configure Notification: There are a couple ways to get notified for these events.

• You can have AWS Lambda subscribed to an Amazon S3 event, such as PutObject which triggers the Lambda as soon as the log file gets delivered to the S3 bucket in Step 1. Lambda can look for the specific events and help you get notified for the event via Amazon SNS topic.
• Another way to get notified is to send CloudTrail events to Amazon CloudWatch Log, have an event rule configured to parse for specific events, and send a notification via Amazon SNS.

You can configure CloudTrail to deliver log files from multiple AWS accounts to a single S3 bucket. You can also leverage the capabilities of Organizations and Delegated Admin to enable and manage CloudTrail.

Security Hub

Security Hub provides a comprehensive view of the security state in AWS and helps check the environment against security industry standards and best practices. It also collects security data from across AWS accounts, services, and supported third-party partner products, and helps analyze trends and identify the highest priority issues. In this post, we leverage the controls of the AWS Foundational Security Best Practices (FSBP) standard in Security Hub, and more specifically those related to Amazon S3 security best practices. You leverage these controls to monitor public S3 buckets.

This solution requires enabling Security Hub and the AWS FSBP standard in the accounts and regions in the Organizations that you want to monitor. Enabling AWS Config recording is a prerequisite for the Security Hub and the AWS FSBP standard. To learn more about how Security Hub runs and uses security checks, refer to this AWS document. Findings from the accounts are shared with the Security Hub administrator account (by default, this is the management account, but administration can be delegated to another member account in the Organization).

You can use Security Hub to detect and alert on public S3 buckets by conducting the following steps:

1. Create an SNS topic: create an SNS topic and one or more email subscriptions for this topic.
2. Create an EventBridge rule: create the following rule that identifies findings related to the following two AWS FSBP controls: “S3 buckets should prohibit public read access” (S3.2) and “S3 buckets should prohibit public write access” (S3.3).

   {
     "source": ["aws.securityhub"],
     "detail-type": ["Security Hub Findings - Imported"],
     "detail": {
       "findings": {
         "Compliance": {
           "Status": ["FAILED"]
         },
         "RecordState": ["ACTIVE"],
         "Workflow": {
           "Status": ["NEW"]
         },
         "ProductFields": {
           "ControlId": ["S3.2", "S3.3"]
         }
       }
     }
   }
   

3. Create an email target for the EventBridge rule: select the SNS topic created in Step 1 as a target for the rule. This allows findings to be sent to the email addresses configured in the topic subscriptions.

With cross-Region aggregation, you can aggregate findings, insights, control compliance statuses, and security scores from multiple Regions to a single aggregation Region. Then, you can manage this data from the aggregation Region.

Amazon GuardDuty

Amazon GuardDuty monitors accounts, users, instances, containers, and storage for potential threats. It uses data sources, such as AWS CloudTrail data events for Amazon S3 logs, CloudTrail management event logs, DNS logs, Amazon Elastic Block Store (Amazon EBS) volume data, Kubernetes audit logs, and Amazon Virtual Private Cloud (Amazon VPC) flow logs. Detection is performed through a combination of anomaly detection, machine learning (ML), and threat feeds to help identify threats.

A GuardDuty finding represents a potential security issue detected within your account. GuardDuty uses the following two findings to monitor S3 buckets with public access: Bucket Public Access Granted and Bucket Anonymous Access Granted. GuardDuty sends findings to Security Hub for the supported findings and this enables you to manage your security events from one location.

The following screenshot illustrates S3/BucketAnonymousAccessGranted finding.

GuardDuty alerts you of the status of your AWS environment that you can view in the GuardDuty console. GuardDuty also lets you export your findings to an Amazon S3 bucket and integrate with other AWS services.

Amazon Macie

Amazon Macie uses ML and pattern matching to discover and report on sensitive data in your accounts. Amazon Macie can be used to analyze your Amazon S3 security posture as well. Once you enable Macie, it discovers and maintains a complete inventory of your Amazon S3 buckets in the Region. Amazon Macie monitors and evaluates the S3 buckets for security and access control.

Amazon Macie generates a policy finding when the policies or settings for an S3 bucket are changed in a way that reduces the security or privacy of the bucket and the bucket’s objects. The following policy S3BucketPublic generates a finding when the bucket configuration is changed to allow access by anonymous users or all authenticated AWS Identity and Access Management (IAM) identities. The following policy S3BlockPublicAccessDisabled generates a finding when the Block all public access is turned off on an S3 bucket.

The following screenshot illustrates S3BucketPublic finding.

If your AWS environment has multiple accounts, you can associate the Amazon Macie accounts in your environment and centrally manage them in Macie with Organizations. To support integration with other monitoring or event management systems, Amazon Macie automatically publishes policy and sensitive data findings to EventBridge. For additional support, you can configure Macie to also publish policy and sensitive data findings to Security Hub. You can learn more about how Amazon Macie publishes findings to Security Hub here.

AWS Trusted Advisor

AWS Trusted Advisor provides recommendations across multiple categories, including cost optimization and security. These recommendations help you optimize your AWS infrastructure, improve security and performance, reduce costs, and monitor service quotas. You can enable Trusted Advisor for your AWS Account from the Manage Trusted Advisor console. Once enabled, you can use the Trusted Advisor console to review results for your AWS account and then follow the recommended steps to fix identified issues.

The Trusted Advisor Organizational view lets you monitor/detect checks for accounts in your Organizations. After you enable this feature, you can create reports to aggregate the check results for the member accounts in your Organization. Trusted Advisor automatically refreshes your checks if you have a Business or Enterprise Support plan. For other support plans, you can manually refresh your checks to receive the latest results.

Trusted Advisor checks Amazon S3 Bucket Permissions, which detects S3 buckets that have open access permissions, or that allow access to authenticated AWS users. This check examines explicit bucket permissions, as well as bucket policies that might override those permissions. You can use EventBridge to invoke one or more target actions when the checks for Trusted Advisor change status. When Trusted Advisor refreshes your checks, it publishes metrics about your check results to CloudWatch. The following are the alert types generated during the check:

• Yellow: The bucket ACL allows List access for Everyone or Any Authenticated AWS User.
• Yellow: A bucket policy allows any kind of open access.
• Yellow: A bucket policy has statements that grant public access.
• Yellow: Trusted Advisor does not have permission to check the policy, or the policy could not be evaluated for other reasons.
• Red: The bucket ACL allows upload and delete access for Everyone or Any Authenticated AWS User.

The following screenshot illustrates Amazon S3 Bucket Permissions check with a Policy Allows Access as ‘Yes’.

From the previous screenshot, you can identify the list of S3 buckets that are publicly accessible due to ACLs or policies that allow read/write access for any user. Unless you intend to have your S3 buckets publicly accessible, you should configure the bucket level S3 Block Public Access feature.

Conclusion

In this post, we covered seven different AWS services that you can use to detect public Amazon S3 buckets across different Regions in your Organization. Companies of various sizes can adopt one of these approaches to detect public S3 buckets as part of their broader cloud security optimization strategy. The right mechanism depends on considerations such as deployment complexity, multi-account and/or multi-Region support, notification, and cost.

Securing data from unintended access is crucial, and the methods featured in this post can help you ensure you aren’t unintentionally exposing or sharing data publicly. As a reminder, IAM Access Analyzer for S3 is available at no additional cost in the S3 console. Be sure to check out the respective pricing pages for each service option mentioned in this post before getting started.

For more information, refer to the AWS Well-Architected Framework, Architecture Best Practices for Storage, and Security best practices for Amazon S3.

Mourad Cherfaoui (Sr. Security consultant) contributed to this post.