Temporarily block data transfers between AWS Regions in Amazon S3

Certain organizations may experience unexpected or outlier cross-region data transfer charges and require time to identify which of their systems or workloads are initiating them. In such cases, it may be useful to temporarily block data transfers to within a particular region. This temporary restriction can prevent further unwanted charges and allows time for audit workflows and investigation of relevant logs to identify offending systems before implementing desired access control policies.

Amazon S3 offers versatile bucket policies that provide granular access controls to your resources. By using the NotIpAddress condition, you can effectively restrict specified AWS Regions from accessing your Amazon S3 resources by blocking their associated IP addresses. This ensures that only desired AWS Regions can ingress and egress data. This technique uses Amazon’s publicly available IP address list, which contains IP addresses by region, to specify non-allowed AWS Regions, thereby blocking cross-Region access to your data for all AWS services in that Region.

In this blog post, I walk through configuring an S3 bucket policy that enforces IP-based restrictions. This solution is for temporarily turning off cross-Region ingress and egress to your bucket so that users are not able to ingress and egress data to and from your bucket from another Region. This limits cross-Region data transfer charges from users who ingress and egress data from buckets that are not in the same Region as their systems. After you have paused transfers between Regions, you can further investigate the original transfers that warranted the temporary restrictions using a variety of AWS logging options available, which I discuss at a high level.

Solution overview

This solution works by blocking all IP addresses in a given Region, which you’ve identified as the Region in which unwanted requests are originating, from accessing the Region in which the policy is put into place. This solution blocks the system that is ingressing or egressing data cross-Region from doing so by returning a 403 error. The following diagram shows the flow of data after you have put the policy into place, with all AWS services from one Region blocked from accessing data in another via IP-based restrictions:

To temporarily block the ingress and egress of a bucket by AWS Region, you are going to use the condition key “NotIPAddress” in the bucket policy. A condition key in an AWS Identity and Access Management (IAM) policy allows you to further refine the permissions granted or denied by the policy based on metadata about the request. A not condition key inverts the meaning of the condition, in our use case, ingress and egress to IP addresses that are not in the NotIPAddress list are restricted. NotIPAddress is an AWS‐wide key that is supported by the AWS services that support policies. You use the NotIPAddress condition to limit calls to a bucket to IP addresses that are not in the IP address range for the AWS Regions you intend to block from making calls to Amazon S3.

AWS publishes its current IP address ranges by Region in the JSON format. You can get the list of IP addresses by Region from aws-ip-ranges. With this information, you can identify traffic from AWS Regions. You can also use this information to allow or deny traffic to or from some AWS services. IP addresses change frequently, so you should only use the solution discussed here to temporarily block traffic from an AWS Region.

Another option, outside the scope of this post, that is more suitable for long-term or permanent blocking of Region-to-Region transfers is using service control policies (SCP), which you can do if the accounts in question are in an AWS Organization. You can read more about using SCPs to block Region-to-Region transfers in this prescriptive guidance.

Prerequisites

For this walkthrough, you should have the following prerequisites:

• An AWS account
• Access to an S3 bucket
• Basic knowledge of how IAM policies work

Walkthrough

To limit ingress and egress by Region you will create a policy on the S3 bucket that blocks IPs from a given AWS Region by populating the NotIPAddress with addresses found in aws-ip-ranges.

1. In your browser navigate to the S3 bucket to which you want to limit ingress and egress.

2. Select the Permissions tab.

3. Scroll down to the bucket policy and select Edit.

4. Add the NotIPAddress condition to the policy, and add the IP addresses from ip-ranges.json for the IP ranges you’d like to block to the condition. You can learn more about the format of ip-ranges.json in the documentation. The following is a templated policy, replace <BUCKET_ARN> with your bucket’s ARN, and replace <IP_ADDDRESSES_From_AWS_IP_RANGES> with the IPs you have identified from regions in the ip-ranges.json file.

{
  "Id": "BlockAccessFromRegions",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "SourceIP",
      "Action": "s3:*",
      "Effect": "Deny",
      "Resource": [
        "<BUCKET_ARN>",
        "<BUCKET_ARN>/*"
      ],
      "Condition": {
        "NotIpAddress": {
          "aws:SourceIp": [
            <IP_ADDRESSES_From_AWS_IP_RANGES>
          ]
        }
      },
      "Principal": "*"
    }
  ]
}

5. You may find this Python script handy to read the aws-ip-ranges and produce a policy for your bucket, thereby limiting ingress and egress from the Regions listed in the “Regions” array. This is just a more convenient way to create the preceding policy prefilled with the correct IP addresses.

import requests
import json

IP_RANGES_URL = "https://ip-ranges.amazonaws.com/ip-ranges.json"
REGIONS = ["us-east-1"]
BUCKET_ARN = "arn:aws:s3:::DOC-EXAMPLE-BUCKET"
POLICY_TEMPLATE = """
{
  "Id": "BlockAccessFromRegions",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "SourceIP",
      "Action": "s3:*",
      "Effect": "Deny",
      "Resource": [
        "<BUCKET_ARN>",
        "<BUCKET_ARN>/*"
      ],
      "Condition": {
        "NotIpAddress": {
          "aws:SourceIp": [
            <IP_ADDRESSES>
          ]
        }
      },
      "Principal": "*"
    }
  ]
}
"""

data = json.loads(requests.get(IP_RANGES_URL).text)
ipAddresses = ""
for i in data['prefixes']:
    if i.get("region") in REGIONS:
        ipAddresses += '"' + i.get("ip_prefix") + '",'

POLICY_TEMPLATE = POLICY_TEMPLATE.replace("<IP_ADDRESSES>", ipAddresses)
POLICY_TEMPLATE = POLICY_TEMPLATE.replace("<BUCKET_ARN>", BUCKET_ARN)

print(POLICY_TEMPLATE)

Investigating the source of unwanted inter-Region transfers

After turning off the regional access, view access logs from integrated services such as Amazon VPC Flow Logs, AWS WAF logs, AWS CloudTrail logs, or application-level logs. These logs can provide insight into the traffic patterns and help identify the source of the cross-region requests.

Based on the information gathered from logs, identify the specific systems or applications in the Region that you have blocked that are making unauthorized requests. Isolate these systems to prevent further cross-region data transfers. Once the offending systems are identified, implement more specific access policies tailored to those systems. For example, you can use IAM policies to restrict access based on specific users, roles, or services involved. Ensure that only authorized entities have the necessary permissions to access the S3 bucket within the allowed region. Logs take time to deliver and analyze, so, as presented in this post, you may want to pause traffic until then to prevent unauthorized transfers and further charges.

Cleaning up

There is no cost associated with attaching policies to a bucket. Therefore, there is no cost to continue to leave this policy in place. However, if this policy is left in place, IPs in the policy will remain blocked. To re-allow ingress and egress from these AWS Regions, simply remove the NotIPAddress condition.

Note that IP-based filtering is not a long-term solution, as IP addresses change regularly. You can use this policy to block cross-Region ingress and egress temporarily. However, due to the fact that IP blocks can change regularly, this solution is a short-term solution to block cross-Region traffic in a situation where you need time to implement audit workflows or investigate logs before correcting the unwanted behavior.

Conclusion

You can block ingress and egress to an S3 bucket from specific AWS Regions by not allowing IP addresses in target Regions using the NotIpAddress condition in tandem with the ip-ranges.json. This method is useful for maintaining control over data ingress and egress temporarily, until you have had time to implement audit workflows or investigate logs to identify the systems or workloads making the unexpected or outlier requests. The flexibility of Amazon S3’s policy system allows for dynamic adjustments as your organizational needs change.

For more detailed guidance on configuring other S3 bucket policies or to deepen your understanding of Amazon S3, visit the Amazon S3 User Guide.