Introduction

As organizations adopt multi-account strategies for improved security features and governance, AWS CloudFormation StackSets enables organizations to deploy infrastructure across multiple accounts and regions. However, monitoring and tracking these distributed deployments across multiple accounts presents operational challenges. When a critical security baseline deployed across 50 accounts suddenly starts failing, teams face the daunting task of logging into each account individually to understand what went wrong and which accounts were affected.

This operational overhead scales exponentially with organization growth, requiring platform teams to spend countless hours switching between accounts and manually correlating deployment events. The lack of centralized visibility slows incident response and makes it difficult to identify patterns or implement proactive monitoring. In this blog post, we’ll explore a solution that centralizes AWS CloudFormation logs from multiple accounts into a single management account, making it easier to monitor and troubleshoot StackSets deployments.

Solution Architecture

Our solution creates a centralized logging system that collects AWS CloudFormation events from all target accounts and forwards them to a central management account. This approach provides a single pane of glass for monitoring and troubleshooting AWS CloudFormation deployments across your entire organization.

Figure 1. Architecture diagram showing event flow from member accounts to management account through EventBridge and CloudWatch Logs.

The architecture consists of four main components:

1. Management Account Setup: Creates a central event bus, log group, and necessary permissions in the organization’s management account.
2. Target Account Configuration: Deployed via StackSets to configure event rules that forward AWS CloudFormation events to the management account.
3. Resource Deployment: Uses StackSets to deploy common resources across target accounts, generating the events we want to monitor.
4. Monitoring and Visualization: Provides dashboards and queries for operational insights.

How It Works

The solution follows this event flow:

1. Event Generation: AWS CloudFormation operations in target accounts generate events (stack creation, updates, deletions, resource changes).
2. Event Capture: Amazon EventBridge rules in each target account capture these AWS CloudFormation events based on defined patterns.
3. Cross-Account Forwarding: Events are forwarded to a custom event bus in the management account using cross-account permissions.
4. Centralized Logging: The central event bus routes all events to a Amazon CloudWatch Log Group with structured logging.
5. Monitoring and Alerting: Administrators can view consolidated logs, create custom queries, and set up alerts from a single location.

Prerequisites

Before implementing this solution, ensure you have the following prerequisites in place:

• AWS account: Ensure you have valid AWS account.
• AWS Organizations: You must have an AWS Organization structure set up with a primary management account and several member accounts under the management account.
• Trusted Access: Enable trusted access for AWS CloudFormation StackSets from the management account (this allows StackSets to assume roles in member accounts).
• Appropriate Permissions: You must have access to the management account or be configured as a delegated administrator to create and manage StackSets. For detailed information about permissions and security considerations when using StackSets with AWS Organizations, please review the Prerequisites in the AWS CloudFormation StackSets documentation.

Implementation Deep Dive

The solution is implemented using two AWS CloudFormation templates that work together to create a comprehensive monitoring system:

1. Management Account Logging Setup (log-setup-management.yaml)

This template establishes the central logging infrastructure in the management account by creating a custom Amazon EventBridge event bus with cross-account access policies and an encrypted Amazon CloudWatch Log Group using a customer-managed AWS Key Management Service (AWS KMS) key. A key feature is the included stack set resource that automatically deploys the target account configuration to all member accounts, eliminating manual setup and ensuring consistent configuration across the entire organization.

2. Stack set Deployment Template (common-resources-stackset.yaml)

This template creates a service-managed stack set that deploys common resources to all accounts in specified organizational units. The StackSet is configured with auto-deployment enabled to automatically provision new accounts added to the organization and includes operation preferences for parallel regional deployment with fault tolerance settings.

Step-by-Step Deployment Guide

Step 1: Download the templates:

• log-setup-management.yaml
• common-resources-stackset.yaml

Step 2: Deploy the Management Account Infrastructure

Deploy the centralized logging infrastructure to your management account.

Using CLI:

aws cloudformation deploy \
  --template-file log-setup-management.yaml \
  --stack-name log-setup-management \
  --parameter-overrides \
    OUID=your-organizational-unit-id \
    OrgID=your-organization-id \
  --capabilities CAPABILITY_IAM \
  --region us-east-1

AWS CLI command execution for stack deployment

Using AWS Console:

1. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.
2. On the Stacks page, choose Create stack at top right, and then choose With new resources (standard).
3. On the Create stack page, Upload a template file, choose Choose File to choose a template file from your local computer.
4. Choose Next to continue and to validate the template.
5. On the Specify stack details page, type a stack name in the Stack name box.
6. In the Parameters section, specify values for the parameters that were defined in the template.
7. Choose Next to continue creating the stack.
8. Acknowledge capabilities and transforms.
9. Choose Next to continue.
10. Choose Submit to launch your stack.

This single deployment:

1. Creates the central logging infrastructure in the management account.
2. Automatically deploys Amazon EventBridge rules to all accounts in the specified OU.
3. Sets up the necessary IAM roles and policies for cross-account access.

Figure 2.1: Screenshot showing successful deployment of log-setup-management.yaml template in the management account

Figure 2.2: Deployment timeline view of log-setup-management.yaml template in the management account

Step 3: Deploy Common Resources

Deploy the sample common resources to demonstrate the logging functionality.

Using CLI:

aws cloudformation deploy \
  --template-file common-resources-stackset.yaml \
  --stack-name common-resources-stackset \
  --parameter-overrides \
    OUID=your-organizational-unit-id \
  --capabilities CAPABILITY_IAM \
  --region us-east-1

AWS CLI command execution for stack deployment

Using AWS Console:

1. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.
2. On the Stacks page, choose Create stack at top right, and then choose With new resources (standard).
3. On the Create stack page, Upload a template file, choose Choose File to choose a template file from your local computer.
4. Choose Next to continue and to validate the template.
5. On the Specify stack details page, type a stack name in the Stack name box.
6. In the Parameters section, specify values for the parameters that were defined in the template.
7. Choose Next to continue creating the stack.
8. Acknowledge capabilities and transforms.
9. Choose Next to continue.
10. Choose Submit to launch your stack.

This creates a stack set that deploys Amazon Simple Storage Service (Amazon S3) infrastructure to all target accounts, generating AWS CloudFormation events that will be captured by your centralized logging system.

Figure 3: Screenshot showing successful deployment of common-resources-stackset.yaml template for target accounts

Step 4: Validation and Testing

Confirm event flow and monitoring functionality by viewing the log streams in the ‘central-cloudformation-logs’ log group.

Monitoring and Visualization

The centralized logging solution provides advanced monitoring capabilities through Amazon CloudWatch Logs Insights and custom dashboards.

You can customize your queries to get:

• Recent AWS CloudFormation events across all accounts.
• Failed stack operations for quick troubleshooting.
• Successful deployments for verification.
• Event distribution by account and region.
• Status breakdown of all AWS CloudFormation operations.

The following query helps you analyze CloudFormation events across your organization by showing:

• Timestamp of events
• Account ID where the event occurred
• Region of deployment
• Resource types being deployed
• Deployment status
• Logical resource identifiers

fields @timestamp, account, region
| parse @message /"resource-type":"(?<resource_type>[^"]+)"/ 
| parse @message /"status":"(?<status>[^"]+)"/ 
| parse @message /"logical-resource-id":"(?<logical_resource_id>[^"]+)"/ 
| sort @timestamp desc

Figure 4: CloudWatch Logs Insights query results showing CloudFormation events across accounts

You can customize your queries to filter for specific conditions such as failed deployment status, particular resource types, or specific accounts to quickly identify and troubleshoot issues across your organization’s AWS CloudFormation deployments.

Cost Implications

When implementing this centralized monitoring solution, you should consider the following cost components:

• Amazon EventBridge pricing – Costs associated with events being published across accounts to the central event bus
• Amazon CloudWatch pricing – Storage costs for the centralized log group storing CloudFormation events from all accounts. Query costs when analyzing the centralized logs
• AWS Key Management Service pricing – Costs related to the customer-managed key used for log encryption

Clean up

To clean up the resources created in this solution, follow these steps:

1. First, delete the common resources stack set (common-resources-stackset) from the AWS CloudFormation console in your management account. This will remove all the resources deployed across your member accounts.
2. After the stack set operations are complete, delete the management account logging setup stack (log-setup-management) to remove the centralized logging infrastructure, including the event bus, log groups, and associated IAM roles.

Note: Make sure all stack set operations are complete before deleting the management account logging setup to ensure proper cleanup of all resources.

Conclusion

Managing infrastructure across multiple AWS accounts doesn’t have to be complex. By centralizing AWS CloudFormation logs, you can gain visibility into your multi-account deployments, troubleshoot issues more efficiently, and help achieve consistent resource deployment across your organization.

This solution demonstrates how AWS services like AWS CloudFormation StackSets, Amazon EventBridge, and Amazon CloudWatch Logs can be combined to create a powerful monitoring system for your infrastructure as code deployments.

Get started today by implementing this solution in your AWS Organization to gain immediate visibility into your multi-account deployments. Download the templates from our GitHub repository and follow the step-by-step guide to enhance your cloud operations.

Authors: