Managing AWS Organizations using the open source org-formation tool — Part 3

This article is a guest post from Olaf Conijn, the creator of org-formation.

• Part 1: Managing AWS Organizations resources using infrastructure as code
• Part 2: Integrating management of resources across accounts using task files
• Part 3: Deploying CloudFormation resources to multiple accounts using Organization Bindings
• org-formation on GitHub

In the first two parts of this series, we learned how to manage your AWS Organizations using Infrastructure as Code (IaC) (Part 1) and how to create a continuous deployment pipeline for changes to your Organizations (Part 2). In the final installment of this series, we will look at org-formation-specific extensions to the AWS CloudFormation IAC language that make AWS CloudFormation aware of the organizational context within the AWS account. We’ll also create references to other resources across different AWS accounts and regions.

Org-formation Annotated CloudFormation templates

Another feature of org-formation is the ability to add organization-aware annotations to regular CloudFormation templates. Regular CloudFormation has no knowledge of organization resources and only supports specifying resources within a template that all need deployment to the same target account and region.

For each individual resource within a template, org-formation allows us to specify which account and region to deploy the resource. The mechanism by which we specify where to deploy a resource is the same Organization Binding as used within a tasks file. This means that resources within a template can be bound to multiple account/region combinations (e.g., by specifying the binding Account: '*').

Note that this is different from CloudFormation StackSets. With the StackSet feature of CloudFormation, you can execute a template in different target accounts and regions. The template, however, will always be the same for all targets. In practice, this means that for any unique set of resources, you must create a new CloudFormation template, resulting in a lot of work spent managing the relationships between these templates.

When executing the org-formation update-stacks command or adding an update-stacks task to a task file, org-formation will generate a CloudFormation template for each target you specified within your bindings. It will also create the resources bound to that target using CloudFormation.

\> org-formation update-stacks template.yml --stack-name my-stack

The following is an example of an Annotated CloudFormation template:

AWSTemplateFormatVersion: '2010-09-09-OC'

# Include the file that contains the Organization Section.
# The Organization Section describes Accounts, Organizational Units, etc.
Organization: !Include ../organization.yml

# Any Binding that does not explicitly specify a region will default to this.
# Value can be either string or list
DefaultOrganizationBindingRegion: eu-central-1

# Bindings determine what resources are deployed where
# These bindings can be !Ref'd from the Resources in the resource section
# Any Resource that does not specify a binding will use this binding.
# This specific binding selects all accounts from your organization that have a budget-alarm-threshold tag.
DefaultOrganizationBinding:
  AccountsWithTag: budget-alarm-threshold

Resources:

  Budget:
    Type: AWS::Budgets::Budget
    Properties:
      Budget:
        BudgetName: !Sub 'budget-${AWSAccount.Alias}' # AWSAccount.Alias resolves to IAM Alias of current account
        BudgetLimit:
          Amount: !GetAtt AWSAccount.Tags.BudgetAlarmThreshold # Resolves to value of tag of current account
          Unit: USD
        TimeUnit: MONTHLY
        BudgetType: COST
      NotificationsWithSubscribers:
        - Notification:
            NotificationType: FORECASTED
            ComparisonOperator: GREATER_THAN
            Threshold: 1
          Subscribers:
            - SubscriptionType: EMAIL
              Address: !GetAtt AWSAccount.Tags.AccountOwnerEmail

The previous template creates a Budget resource for every account in the organization with a tag BudgetAlarmThreshold. In the properties of this resource, various references to the organization.yml file are used:

• The BudgetName of the Budget resource is a composite of budget and the value of the IAM alias in the created account. This is useful for identifying to which AWS account a Budget notification applies.
• The Amount of the BudgetLimit specifies the value of the tag BudgetAlarmThreshold of the Budget resource in the created account.
• The Address of the Email Subscriber specifies the value of the tag AccountOwnerEmail of the Budget resource in the created account.

Note that when resolving these references, the values are read from the organization.yml file that is included either by Organization attribute, or by the tasks file. If we manually change the value of the tag in the AWS console, org-formation will not know. If we change the value of a tag in the organization.yml, then org-formation knows that it needs to run both update-organization and update-stacks for templates that reference tags. Also, the Organization attribute does not require specification when including a template from within a tasks file. Overwrite attributes like DefaultOrganizationBindingRegion and the bindings from within a tasks file.

A reference to AWSAccount will resolve to the account the CloudFormation template executes in, much like AWS::AccountId. However, we can refer to any account in the organization.yml file by its logical name (e.g., !GetAtt MyDevAccount.Tags.AccountOwnerEmail or !Ref MyDevAccount) are also valid expressions, assuming we declared an account named MyDevAccount.

Cross account references in CloudFormation templates

As org-formation templates contain resources that will be deployed to multiple accounts, they can also contain the relationships (!Ref or otherwise) between these resources.

For example:

AWSTemplateFormatVersion: '2010-09-09-OC'

# Include the file that contains the Organization Section.
# The Organization Section describes Accounts, Organizational Units, etc.
Organization: !Include ../organization.yml

# Any Binding that does not explicitly specify a region will default to this.
# Value can be either string or list
DefaultOrganizationBindingRegion: eu-central-1

# Section that contains a named list of Bindings.
# Bindings determine what resources are deployed where
# These bindings can be !Ref'd from the Resources in the resource section
OrganizationBindings:

  # Binding for: S3Bucket, S3BucketPolicy
  CloudTrailBucketBinding:
    Account: !Ref ComplianceAccount

  # Binding for: CloudTrail
  CloudTrailBinding:
    Account: '*'
    IncludeMasterAccount: true

Resources:

  S3Bucket:
    OrganizationBinding: !Ref CloudTrailBucketBinding
    DeletionPolicy: Retain
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub 'cloudtrail-${ComplianceAccount}'

  S3BucketPolicy:
    OrganizationBinding: !Ref CloudTrailBucketBinding
    Type: AWS::S3::BucketPolicy
    DependsOn: S3Bucket
    Properties:
      Bucket: !Ref S3Bucket
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Sid: 'AWSCloudTrailAclCheck'
            Effect: 'Allow'
            Principal: { Service: 'cloudtrail.amazonaws.com' }
            Action: 's3:GetBucketAcl'
            Resource: !Sub 'arn:aws:s3:::${CloudTrailS3Bucket}'
          - Sid: 'AWSCloudTrailWrite'
            Effect: 'Allow'
            Principal: { Service: 'cloudtrail.amazonaws.com' }
            Action: 's3:PutObject'
            Resource: !Sub 'arn:aws:s3:::${CloudTrailS3Bucket}/AWSLogs/*/*'
            Condition:
              StringEquals:
                s3:x-amz-acl: 'bucket-owner-full-control'

  CloudTrail:
    OrganizationBinding: !Ref CloudTrailBinding
    Type: AWS::CloudTrail::Trail
    DependsOn:
      - CloudTrailS3BucketPolicy
    Properties:
      S3BucketName: !Ref S3Bucket
      IsLogging: false
      IncludeGlobalServiceEvents: true
      IsMultiRegionTrail: true

The previous example demonstrates a CloudFormation template with three resources: CloudTrail, S3Bucket, and S3BucketPolicy. The CloudTrail resource deploys to all accounts, and the S3Bucket and S3BucketPolicy will only generate in the ComplianceAccount.

Executing org-formation creates a template for every account in the organization (the CloudTrail resource is bound to all accounts). All these templates will contain a CloudTrail resource. The template created for the ComplianceAccount will additionally contain the S3Bucket and S3BucketPolicy resources.

The CloudTrail resource has a reference to the S3Bucket resource, which is bound only to the ComplianceAccount account. What org-formation will do for all accounts that do not have both resources is create a CloudFormation export in the template deployed to the ComplianceAccount and declare a parameter in the templates deployed to all other accounts. When deploying, org-formation will create a dependency between the templates, to ensure the right order of execution, and copy the value from the export into the parameter of the other templates when deploying these.

This example illustrates the fragments from deploying the template to the ComplianceAccount:

Resources:
  S3Bucket:
    DeletionPolicy: Retain
    Type: AWS::S3::Bucket
    Properties:
      BucketName: cloudtrail-111111111111

# ... S3BucketPolicy omitted ....

# Output section generated by org-formation for template deployed to the ComplianceAccount
Outputs:
  printDashCloudTrailS3Bucket:
    Value: !Ref S3Bucket
    Description: Cross Account dependency
    Export:
      Name: mystackname-CloudTrailS3Bucket

The cross account expression (!Ref S3Bucket) will be copied to the Value of the output. This can be any expression, also !GetAtt or !Sub.

This example illustrates the fragments from the template that will be deployed all accounts, except for the ComplianceAccount:

Parameters:
  CloudTrailS3Bucket:
    Description: Cross Account dependency
    Type: String
    ExportAccountId: '340381375986'
    ExportRegion: eu-central-1
    ExportName: mystackname-CloudTrailS3Bucket

 # ... further down in the Resources section ...

 CloudTrail:
  Type: AWS::CloudTrail::Trail
  Properties:
    S3BucketName:
      Ref: CloudTrailS3Bucket

Note the removal of the DependsOn attribute from the original template. Although org-formation understands the relationship between the templates, CloudFormation does not, and there is no use for the DependsOn within the template deployed to CloudFormation. Being able to use references to organization resources and resources bound to different accounts allows us to create templates. These templates describe how to apply entire best practices and patterns to a multi-account setup. References also allow us to re-use these templates, as they do not contain account IDs or require you to deploy multiple CloudFormation templates.

Additional CloudFormation Annotations

Once we start modelling different parts of our resource baseline in CloudFormation, we will notice that we might need more than just the ability to refer to organization resources or resources across accounts/regions. Other features that can be useful are:

• ForeachAccount attribute: Specifying a binding as the value of this attribute will create a copy of the resource for each account in the binding. This can be useful when setting up host names and certificates in our MasterAccount for each account that needs one, or when implementing Amazon GuardDuty and applying this to all accounts in our organization.
• Fn::EnumTargetAccounts function: This function allows us to create an array of values for each account in a binding. Use this when setting up cross account IAM permissions that adhere to the principle of least privilege.

This is an example of the use of ForeachAccount:

Member:
  Type: AWS::GuardDuty::Member
  OrganizationBinding:
      IncludeMasterAccount: true
  ForeachAccount:
      Accounts: '*'
  Properties:
    DetectorId: !Ref Detector
    Email: !GetAtt CurrentAccount.RootEmail
    MemberId: !Ref CurrentAccount
    Status: Invited
    DisableEmailNotification: true

In the example, a Member resource generates for each account in the specified binding (Accounts: '*'). When creating a resource for each account in the binding, useCurrentAccount to resolve information about the account being iterated over. AWSAccount will still refer to the AWS account part of the target (in this case the MasterAccount). A full example on how to implement GuardDuty using org-formation can be found in GitHub.

This is an example of Fn::EnumTargetAccounts, and how to create a resource policy and provide access to other accounts:

OrganizationBindings:
  # Binding for: Bucket, BucketPolicy
  BucketAccountBinding:
    Account: !Ref MyAccount

  # Binding for: S3BucketReadAccessPolicy
  ReadAccessAccountBinding: # default = empty binding

Conditions:
  CreateReadBucketPolicy: !Not [ !Equals [ Fn::TargetCount ReadAccessAccountBinding, 0 ] ]

Resources:
  Bucket:
    Type: AWS::S3::Bucket
    OrganizationBinding: !Ref BucketAccountBinding
    DeletionPolicy: Retain
    Properties:
      BucketName: !Sub '${bucketName}'

  BucketReadPolicy:
    Type: AWS::S3::BucketPolicy
    OrganizationBinding: !Ref BucketAccountBinding
    Condition: CreateReadBucketPolicy
    Properties:
      Bucket: !Ref Bucket
      PolicyDocument:
        Statement:
          - Sid: 'Read operations on bucket'
            Action:
            - s3:Get*
            - s3:List*
            Effect: "Allow"
            Resource:
            - !Sub '${Bucket.Arn}'
            - !Sub '${Bucket.Arn}/*'
            Principal:
              AWS: Fn::EnumTargetAccounts ReadAccessAccountBinding arn:aws:iam::${account}:root

In this example, we created a Bucket resource in MyAccount. A BucketPolicy provides Get/List access to this bucket for all the accounts that are part of the ReadAccessAccountBinding. In the same example, the task file supplies the ReadAccountAccountBinding. The default specified in the template is an empty binding (no account will get access to the Bucket resource).

Note that as the default is an empty binding, the EnumTargetAccounts will generate an empty array and it is only possible to create a valid BucketPolicy if there are more than zero accounts part of the ReadAccountAccessBinding. The function Fn::TargetAccount will return the number of accounts part of a binding, which can be used in a CloudFormation condition. We can find a more complete example on how to set up cross account access to Amazon S3 buckets in GitHub.

Summary

In this series, we learned about three features that the org-formation tool provides. Use each of these to set up and manage resources across  AWS Organizations:

• Part 1: Managing AWS Organizations resources using infrastructure as code
• Part 2: Integrating management of resources across accounts using task files
• Part 3: Deploying CloudFormation resources to multiple accounts using Organization Bindings
• org-formation on GitHub

We wrote this article on version 0.9.6 of org-formation. For the most recent version, examples, and documentation, refer to the Github project page.

Feel free to engage, create issues, ask questions over Slack, provide feedback, and share your experiences.

The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.