AWS Cloud Operations Blog

Managing AWS resources across multiple accounts and Regions using AWS Systems Manager Automation

AWS Systems Manager Automation simplifies common administrative and maintenance tasks of AWS resources. Using Systems Manager Automation, you can execute predefined tasks/workflows in the form of AWS Systems Manager documents (SSM documents) that you can write yourself or use community published documents. A SSM document defines the actions that Systems Manager performs on your AWS resources. These documents can be scheduled using Systems Manager Maintenance Windows. Then the documents can be triggered based on changes to AWS resources through Amazon CloudWatch Events, or executed directly through the AWS Management Console, AWS CLI, or AWS SDKs. You can track the execution of each step in the document as well as require approvals for a step. You can also incrementally roll out changes and automatically halt when errors occur.

Many AWS customers use multiple AWS accounts. They often use AWS Organizations to arrange the accounts into a hierarchy and to group them into organizational units (OUs). OUs can be used for variety of purposes such as for consolidated billing, workload isolation, and administrative isolation. Customers often create separate accounts for development, testing, staging, and production on a per-application basis as part of an Organization.

In the multi-account structure as described earlier, there are some repetitive tasks that are common across all the accounts but managing them independently can be challenging. Customers tell us that they want to execute common tasks, such as the following, from one central/management account:

  • Patch Management
  • Creating golden Amazon Machine Images (AMIs)
  • Using SSM Agent to update software agents
  • Starting/stopping/rebooting Amazon EC2 instances
  • Encrypting Amazon S3 buckets

Introducing Multi-account and Multi-Region Automation

To address these customer challenges, we recently launched Systems Manager Automation multi-account and multi-Region support. With this feature, you can now execute your own Automation documents across multiple AWS accounts and/or Regions with a few clicks from a master AWS account. This feature also works with AWS-managed Automation documents.

This feature always works on a cross-account basis. The administrator or master account owns one or more Automation documents and controls deployment to one or more target accounts. The administrator account must include an assumable AWS Identity and Access Management (IAM) role, and the target accounts must delegate trust to the administrator account.

Using Multi-account Automation

Let’s consider the use case in which you want to enable AWS CloudTrail across all accounts and Regions, and you want to store CloudTrail logs in a central Amazon S3 bucket in a master/central AWS account. Centralized CloudTrail logging is a critical requirement for organizations that have a central IT department that supervises compliance and auditing. Following is the diagram representing this architecture.

 

 

You can use the AWS provided document AWS-EnableCloudTrail or you can write a custom Automation document similar to the following that will allow you to create a multi-Region trail in target accounts. This document will create a trail named “Multi-Account-Trail” in all the specified target accounts and Regions to push those logs into a central S3 bucket of your choice.

 

---
description: "Automation Document for Central CloudTrail logging YAML Template"
schemaVersion: "0.3"
assumeRole: "{{ AutomationAssumeRole }}"
parameters:
  S3Bucket:
    type: "String"
    description: "(Required) Central Amazon S3 bucket for storing copies of all AWS CloudTrail from multiple accounts in all Regions"
  S3KeyPrefix:
    type: "String"
    description: "(Optional) Central Bucket Key prefix"
    default: ""
  TrailName:
    type: "String"
    description: "(Optional) Name to be assigned to this trail"
    default: "Multi-Account-Trail"

  MultiRegionTrail:
    type: "Boolean"
    description: 
    default: True
  AutomationAssumeRole:
    type: "String"
    description: "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf."
    default: ""

mainSteps:

- name: CreateTrail
  action: aws:executeAwsApi
  inputs:
    Service: cloudtrail
    Api: CreateTrail
    Name: "{{TrailName}}"
    IsMultiRegionTrail: True
    S3BucketName: "{{ S3Bucket }}"
    S3KeyPrefix: "{{ S3KeyPrefix }}"

- name: StartLogging
  action: aws:executeAwsApi
  inputs:
    Service: cloudtrail
    Api: StartLogging
    Name: "{{TrailName}}"

 

I have created this document in my AWS account as “Centralized-cloudtrail”:

 

 

Your S3 bucket should have a bucket policy similar to this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AWSCloudTrailAclCheck20150319",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudtrail.amazonaws.com"
            },
            "Action": "s3:GetBucketAcl",
            "Resource": "arn:aws:s3:::{BUCKETNAME}"
        },
        {
            "Sid": "AWSCloudTrailWrite20150319",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudtrail.amazonaws.com"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::{BUCKETNAME}/*",
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        }
    ]
}

 

To use the cross-account Automation feature you need to first setup the appropriate roles in both the master and target accounts. In your master account, provision administrator role AWS-SystemsManager-AutomationAdministrationRole and provision execution role AWS-SystemsManager-AutomationExecutionRole in every target account. Step-by-step instructions for creating these roles are provided here.

After the roles have been created, use the cross-account Automation feature to execute the document provided in this blog to enable AWS CloudTrail in each target account and push the logs from all the target accounts to an S3 bucket for centralized log collection and analysis.

The following snapshot shows how you can execute the Centralized-cloudtrail document that we previously created. We select the multi-account and Region execution option, input the target accounts, select the target Region, select a unique trail name (if the chosen name already exists in target account automation will fail), and then enter the name of the S3 bucket where we will receive the CloudTrail logs from target accounts. You can also optionally enter a prefix in the S3 bucket where the logs will be stored. Systems Manager Automation by default uses AWS-SystemsManager-AutomationAdministrationRole to execute the cross-account automation. However you can also optionally choose a different role with necessary permissions.

 

 

After executing the automation, I can verify that it was successful:

In my target account, we can see that Multi-Account-Trail was successfully created:

 

 

Similarly, you can also execute this automation through following CLI command:

aws ssm start-automation-execution —document-name Centralized-cloudtrail —parameters AutomationAssumeRole=arn:aws:iam::<Master_ACCOUNTID>:role/AWS-SystemsManager-AutomationAdministrationRole,S3Bucket=<Central_S3_Bucket_Name> —target-locations Accounts=<Target_AccountIDs>,Regions=us-west-2,ExecutionRoleName=AWS-SystemsManager-AutomationExecutionRole —target-parameter-name TrailName —targets Key=ParameterValues,Values=Multi-Account-Trail Values —region us-west-2

Additional use-case

Another application for this feature is to stop/start non-production EC2 instances in several different accounts to save costs. You can tag all such instances as Name=Non-Prod and then execute Automation document AWS-StartEC2Instance to start the EC2 instances simultaneously in all the target accounts using this tag. Then you can use AWS-StopEC2Instance to stop them later. Many of you are operating on the scale of hundreds or even thousands of instances across several different AWS accounts and Regions. You can imagine how this feature can help you save time and significantly simplify management of AWS resources from one central place.

Here are the AWS CLI commands to perform these operations:

aws ssm start-automation-execution --document-name AWS-StartEC2Instance --parameters AutomationAssumeRole=arn:aws:iam::<Master_ACCOUNTID>:role/AWS-SystemsManager-AutomationAdministrationRole --target-locations Accounts=<Target_AccountIDs>,Regions=us-west-2,ExecutionRoleName=AWS-SystemsManager-AutomationExecutionRole --target-parameter-name InstanceId --targets Key=tag:Name,Values=Non-Prod --region us-west-2
aws ssm start-automation-execution --document-name AWS-StopEC2Instance --parameters AutomationAssumeRole=arn:aws:iam::<Master_ACCOUNTID>:role/AWS-SystemsManager-AutomationAdministrationRole --target-locations Accounts=<Target_AccountIDs>,Regions=us-west-2,ExecutionRoleName=AWS-SystemsManager-AutomationExecutionRole --target-parameter-name InstanceId --targets Key=tag:Name,Values=Non-Prod --region us-west-2

Conclusion

In this blog post, I showed how to use the multi-account multi-region automation feature to enable CloudTrail log collection from multiple AWS accounts in one S3 bucket and listed another example of stopping and starting non-prod EC2 instances across several accounts during idle time to save costs.

 

About the Author

Sohaib Tahir is a Solutions Architect for State and Local Government Public Sector team. Sohaib has a focus in networking and automation. He loves to help customers in building architectures with automation services like AWS Systems Manager, AWS Lambda and CloudFormation.