Transfer AWS accounts between AWS Organizations while preserving AWS Lake Formation permissions 

Many AWS customers move their AWS accounts between organizations When your company manages more than one organization, and whether you regularly move accounts between them; or you are consolidating accounts after a merger, acquisition, or divesture. Account migrations are part of operating on AWS.  

Previously, moving an account meant removing it from the source organization, making it standalone, then inviting it to the target organization. For accounts with AWS Resource Access Manager (AWS RAM) resource shares, there was more to plan for: when an account left an organization, AWS RAM removed that account from the principal list of organization-level resource shares. Production workloads depending on cross-account shared resources needed careful coordination to avoid disruption. 

We’ve launched two features that change how customers move accounts between organizations: 

• AWS Organizations direct account transfers. Accounts can now move directly between organizations with no standalone phase in between. Governance, consolidated billing, and account configurations stay active throughout. 

• AWS RAM resource share retention. A new parameter, RetainSharingOnAccountLeaveOrganization, keeps resource shares in place when accounts change organizations. You can enforce this across your organization with the matching AWS Identity and Access Management (IAM) condition key (ram:RetainSharingOnAccountLeaveOrganization) in a service control policy. 

However, some AWS services coordinate sharing across multiple layers. AWS Lake Formation is an example. When you grant cross-account permissions in Lake Formation, it uses AWS RAM behind the scenes to share AWS Glue Data Catalog resources (databases and tables) across accounts in organizations. So, a single cross-account permission involves Organizations, AWS RAM, Lake Formation, and Glue working together.

In this post, we walk you through how to migrate member accounts between organizations while preserving Lake Formation permissions and data access. We use temporary bridge shares with the new AWS RAM retention parameter to keep access to AWS Glue Data Catalog resources intact, so that queries, ETL jobs, and downstream analytics keep running throughout the migration. 

Solution Overview 

We have built a solution for customers that have shared resources across their organization with AWS Lake Formation and need to move accounts between two organizations without losing user access to the data. 

In our example, we have a management account (A1) in org A that has granted Lake Formation permissions to member accounts A2 and A3. Under the hood, Lake Formation created RAM resource shares in A1 to give accounts A2 and A3, access to Data Catalog databases and tables. When A2 and A3 move to org B, the original AWS RAM shares remove them from their principal lists, and database and table access stop working. 

In our solution, we will create temporary bridge shares in A1 which are copies of the original AWS RAM shares but with RetainSharingOnAccountLeaveOrganization enabled. These bridge shares keep data access intact while the accounts move. Once the migration is done, we restore the original shares with the migrated account IDs and clean up the bridge shares. 

The migration process involves four high-level steps, as illustrated in Figure 1.  

1. Create bridge shares — In the Org A management account (A1), create copies of the existing AWS RAM shares (originally created by Lake Formation) as new bridge shares with special retention parameters enabled. 
2. Migrate member accounts — Move the member accounts (A2, A3) from org A to org B. 
3. Repair original shares — In account A1, restore the principal IDs in the original AWS RAM shares. 
4. Delete bridge shares — In account A1, remove the temporary bridge shares. 

Figure 1. Process for migrating member AWS accounts across organizations while preserving Lake Formation permissions and data access intact. 

The data access created and managed by Lake Formation for the member accounts is preserved throughout all steps. 

Prerequisites 

Before you begin, make sure you have the following: 

1. An IAM role with the following access to the management account in your organization that would allow creating and deleting AWS RAM shares as well as removing the member accounts from the origin organization. 
   1. The AWS managed policy AWSLakeFormationCrossAccountManager. This handles AWS RAM create/update/delete for Lake Formation resources.  
   2. A custom inline policy with organizations:RemoveAccountFromOrganization and organizations:DescribeOrganization 
2. A Lake Formation administrator role configured in the management account (A1). 
3. The Boto3 Python scripts provided for this migration (each script includes a ReadMe file with detailed usage instructions) require Python 3.10 or above and Boto3 version 1.42.66 or above. 
4. IAM Administrator access to the member accounts (A2, A3) to create the required IAM role OrganizationAccountAccessRole. 

Note: Identify all the accounts that have granted Lake Formation permissions to any other account in org A. You must run the scripts in every account sharing a resource. This is because AWS RAM shares are created and maintained in the grantor account for each cross-account permission. 

The management account holds the databases and tables in its Data Catalog as well as Lake Formation permissions. We will run it from the management account. In your environment, the Data Catalog tables and corresponding Lake Formation permissions can be in any member account. Run the relevant scripts described in the below steps from the grantor accounts.  

Walkthrough

We will walk through the steps involved to run the boto3 scripts, which are available for download from the Github Repo sample-aws-ram-org-migration. 

 Step 1: Create the IAM role in member accounts 

Create an IAM role named OrganizationAccountAccessRole in each member account (A2 and A3) that has Lake Formation permissions granted by management account A1. Account A1 assumes this role to accept the new bridge resource share invitations sent to the member accounts. 

Note: The role OrganizationAccountAccessRole may be already present in the member account, if member account was created through the organizations from console or APIs.   

Sample Role ARN:  

arn:aws:iam::<member-account-id>:role/OrganizationAccountAccessRole 

Permissions policy: 

{
   "Version": "2012-10-17", 
   "Statement": [ 
       { 
           "Sid": "AllowBridgeRAMShareAcceptance", 
           "Effect": "Allow", 
           "Action": [ 
               "ram:AcceptResourceShareInvitation" 
           ], 
           "Resource": ["*"], 
           "Condition":{  
               "StringLike": { 
                  "ram:ResourceShareName": "*-bridge" 
               } 
           } 
       }, 
       { 
           "Sid": "AllowRAMShareViewInvitations", 
           "Effect": "Allow", 
           "Action": [ 
               "ram:GetResourceShareInvitations" 
           ], 
           "Resource": ["*"] 
       } 
   ] 
} 

 Trust policy: 

{
   "Version": "2012-10-17", 
   "Statement": [ 
       { 
           "Sid": "AllowManagementAccountAssumeRole", 
           "Effect": "Allow", 
           "Principal": { 
               "AWS": "arn:aws:iam::<accountA-id>:role/<datalake_administrator_role>" 
           }, 
           "Action": "sts:AssumeRole" 
       } 
   ] 
}

Step 2: Create bridge shares  

Bridge shares are copies of the original AWS RAM shares containing the same resources, principal IDs, and managed permissions policies, but with the additional parameter Retain access for accounts that left the organization set to Enabled. These bridge shares ensure that Lake Formation permissions continue to function when member accounts leave Org A. 

In the org A management account (A1), run the create_bridge_shares.py script in dry run mode: 

 python3 create_bridge_shares.py --dry-run 

 Review and verify all details of the AWS RAM shares that will be copied into new bridge shares. 

Once you have verified the bridge share details, run the script in execute mode: 

python3 create_bridge_shares.py --execute 

Step 3: Migrate member accounts 

Migrate member accounts A2 and A3 from org A to org B, following instructions in Migrate an account to another organization with AWS Organizations. 

At this point: 

• Lake Formation permissions remain effective. 

• Member accounts can continue to access tables without interruption. 

• The existing AWS RAM shares that were created by Lake Formation will have the migrating accounts removed from their principal lists. The bridge shares maintain access during this interim period. 

Step 4: Restore principals in original shares 

After the member account migration is complete, return to management account A1. As a Data Lake Administrator, run the restore_principals.py script in dry run mode first: 

 python3 restore_principals.py --dry-run 

The script sets the flag Allow external principals to Yes on the original RAM shares and also adds the migrated member account IDs back to the principal list of the original shares.  

Verify all details, then run it in execute mode: 

python3 restore_principals.py --execute 

Verify that you have data access in the member accounts. 

Step 5: Delete bridge shares 

As a Data Lake Administrator, run the delete_bridge_shares.py script in dry run mode first. 

python3 delete_bridge_shares.py --dry-run 

The script removes the temporary bridge shares, since the original shares have been repaired with the required parameters. 

Verify all details, then run it in execute mode. 

python3 delete_bridge_shares.py --execute   

Verify that member accounts continue to have data access. 

Important Note: Remember to run the scripts in each account in the data mesh that shares its Data Catalog tables and databases using Lake Formation permissions. 

 Post-Migration Validation 

While this migration process is designed to preserve your existing permissions and access controls, we strongly recommend that you validate all permissions, policies, and other configurations after the migration is complete, specifically: 

• Verify that all cross-account Lake Formation permissions are functioning as expected. 

• Confirm that queries, ETL jobs, and analytics pipelines in the migrated accounts can access shared Data Catalog resources. 

• Review AWS RAM share configurations to ensure principal lists and sharing parameters align with your business requirements.

Conclusion 

Using direct account transfers and AWS RAM resource share retention, you can move accounts between organizations without impacting your workloads. In this post, we showed how to apply these features to migrate accounts, while preserving access to Data Catalog tables managed by Lake Formation cross-account permissions. 

The same bridge share pattern of migrating works for other AWS RAM-shared resources in your environment. Whether you are consolidating organizations after an acquisition, splitting them during a divestiture, or just moving accounts between organizations you manage, the approach is the same: create bridge shares with retention enabled, move the accounts, restore the originals, and clean up. 

To learn more, see Granting and revoking cross-account permissions in the Lake Formation Developer Guide and the AWS RAM User Guide.