Managing AWS Organizations using the open source org-formation tool — Part 1

This article is a guest post from Olaf Conijn, the creator of org-formation.

• Part 1: Managing AWS Organizations resources using infrastructure as code
• Part 2: Integrating management of resources across accounts using task files
• Part 3: Deploying CloudFormation resources to multiple accounts using Organization Bindings
• org-formation on GitHub

Over the years, AWS customers have found themselves managing multiple AWS accounts. Reasons for doing so include organic adoption of AWS; incremental, per team/department migration to AWS; mergers and acquisitions; or when creating isolation between applications, sometimes using distinct accounts for development, testing, and production.

AWS Organizations provides a way for customers to manage accounts centrally, allowing them to set and apply policies, set controls, and manage billing and cost management across accounts. AWS Organizations is an AWS service used to create new AWS accounts and associate these with the AWS account used to enable the Organizations service.

Using AWS Organizations to manage multiple AWS accounts has a number of benefits, including:

• Reducing the impact of mistakes or security incidents in accounts
• Scaling in terms of resource limits over different accounts
• Simplifying adherence to security principles, such as least privilege, by placing resources in different accounts

For more information and considerations on how to set up AWS Organizations, read the article “Off to a great start with AWS Organizations.”

AWS Organizations Formation (org-formation) is an open source and community-supported tool that allows users to manage different aspects of their AWS Organizations through Infrastructure as Code (IaC). In this series of posts, we will introduce org-formation and explain how to get started.

The first part of this series explains how to use org-formation to manage AWS Organizations using code. We’ll also cover how to generate a model based on the resources (AWS accounts, for example) we already have in our organization, and how to create new accounts and other AWS Organizations resources.

Part 2 explains how to use AWS CodePipeline to set up automatic deployments to our organization and resources.

Part 3 introduces extensions to AWS CloudFormation that allow us to create templates that are aware of the AWS Organizations structure we have, and help create relationships between resources across different AWS accounts and regions.

Getting started with AWS Organizations Formation

Managing AWS Organizations as code allows us to store the definition (or code) that describes our AWS Organizations in a source code repository and automate the deployment of changes. This will decrease the likelihood of human errors when making changes and improves the auditability of changes made to AWS Organizations.

AWS Organizations Formation supports three main features:

• Managing the AWS Organizations resources as code (for example, creating a new AWS account, organizational unit, or service control policy).
• Annotating AWS CloudFormation templates with organization bindings that describe which AWS CloudFormation resources need deployment, where to deploy them, and the relations between these resources.
• Automated deployment of changes to both our AWS Organizations resources and the annotated CloudFormation templates, such as AWS Cloud Development Kit (AWS CDK) or serverless projects.

Prerequisites

To use AWS Organizations Formation, we need an environment that has the following:

• NodeJS, at least version 8.10
• Administrator access to our AWS account—using an AWS Identity and Access Management (IAM) role, not root account
• AWS Organizations set up in our AWS account, either by enabling the service in the AWS Management Console, or by creating accounts and other resources using another tool.

The first step is to install org-formation using npm with the command:

\> npm i aws-organization-formation -g

Generating an organization.yml file

To start using org-formation, we must create an organization.yml file. The organization.yml file will contain the definition of our AWS Organizations resources. We don’t have to create this file manually; org-formation allows us to generate an organization.yml file using the init command.

We can run the init command regardless of which tool we have used to create the AWS Organizations. We can also run the init command again at a later point in time to recreate a new organization.yml file, if needed.

The following command will generate an organization.yml file for our organization:

\> org-formation init organization.yml --region eu-central-1

Note that the --region option is required, as the init command will create an Amazon Simple Storage Service (Amazon S3) bucket. The --region option specifies which region to create the bucket in.

A --profile option, although not required, references an AWS profile configured in the AWS Command Line Interface (AWS CLI). The AWS credentials used, either the default credential or those associated with the profile specified, must give access to the AWS account that contains the AWS Organizations resources (the master account). The credentials must also have sufficient rights to interact with the AWS Organizations service and assume roles within the accounts contained within the AWS Organizations.

By default, the name for the Amazon S3 bucket used to store state is organization-formation-${masterAccountId}. We can change the name of the bucket (using the --state-bucket-name option) and the name of the file org-formation stores its state (using the --state-object option).

We now have a file called organization.yml in our directory that contains all the different resources currently within AWS Organizations in our master account.

For example:

AWSTemplateFormatVersion: '2010-09-09-OC'
Description: default template generated for organization with main account 111222333444

Organization:
  MasterAccount:
    Type: OC::ORG::MasterAccount
    Properties:
      AccountName: My Organization Master Account
      AccountId: '111222333444'

  OrganizationRoot:
    Type: OC::ORG::OrganizationRoot
    Properties:
      DefaultOrganizationAccessRoleName: OrganizationAccountAccessRole

  ProductionOU:
    Type: OC::ORG::OrganizationalUnit
    Properties:
      OrganizationalUnitName: production
      Accounts: !Ref ProductionAccount

  DevelopmentOU:
    Type: OC::ORG::OrganizationalUnit
    Properties:
      OrganizationalUnitName: development
      Accounts:
        - !Ref DevelopmentAccount

  ProductionAccount:
    Type: OC::ORG::Account
    Properties:
      AccountName: Production Account
      AccountId: '111111111111'
      RootEmail: aws-accounts+production@myorg.com

  DevelopmentAccount:
    Type: OC::ORG::Account
    Properties:
      AccountName: Development Account
      AccountId: '222222222222'
      RootEmail: aws-accounts+dev@myorg.com

In the previous example, we will find the following resources:

• MasterAccount. This resource, of type OC::ORG::MasterAccount, refers to the AWS account that contains the AWS Organizations resources. A MasterAccount resource must be part of the template and it must have an AccountId attribute. The value of this attribute will be compared with the account ID stored in the state file and the AWS account to which we are deploying changes, to prevent updating the wrong AWS account by mistake. Apart from these requirements, the MasterAccount can have all the attributes any other account resource has.
• OrganizationRoot. This resource, of type OC::ORG::OrganizationRoot, is the root object for the hierarchical structure that contains accounts and organizational units. We can use this resource to attach service control policies (SCPs) to all of the accounts within the organization. The OrganizationRoot also contains the name of the IAM role used to log in to the other AWS accounts within the organization. The default is OrganizationAccountAccessRole and can be overridden when running the init command (using the --cross-account-role-name option).
• ProductionOU and DevelopmentOU: These resources, of type OC::ORG::OrganizationRoot, are two organizational units (OUs). In this example, these are contained within the OrganizationRoot. Use Organizational units to contain AWS accounts, other organizational units, and/or apply service control policies to accounts within the organizational unit.
• ProductionAccount and DevelopmentAccount: These resources, of type OC::ORG::Account, are two AWS accounts. In this example, they are contained in the ProductionOU and DevelopmentOU organizational units. The relationship is created by adding a !Ref to the Accounts attribute of the organizational unit.

Unlike the OrganizationRoot and MasterAccount, the number of organizational units and accounts in our organization.yml will depend on the number of organizational units and accounts that exist in our organization when generating the file. We might also have service control policy resources in our organization.yml file if we had these configured in AWS.

Any of these resources might have been created by org-formation or by another tool. Frankly, it doesn’t matter which tool created them—from now on we can use the organization.yml file to manage (create/update/delete) these resources by changing them in the organization.yml file and executing the org-formation update command.

Note that AWS accounts cannot be deleted using an API. If we remove an AWS account from the organization.yml file, it won’t be removed; it will be forgotten. We can restore the account later by adding it to the organization.yml file. Although not present in the organization.yml file, the account cannot be used as a reference. If we want to delete an AWS account, we must log in as root and delete the account from within the console.

Updating AWS Organizations resources

If we have an organization.yml file that describes our AWS Organizations resources, we can make changes to these resources and run the update command to apply these changes to the AWS Organizations in our main account. To do so, change the file and run the org-formation update command:

\> org-formation update organization.yml

Note that:

• The --region option is not there, as no regional resources will be created. (AWS Organizations is only available in us-east-1.)
• Prior to update, an init command will create the state bucket. If we provided a --state-bucket-name (or --state-object) option to the init command, we must pass these options to the update command as well.

Having different state buckets (or state objects) can be a good idea when testing changes to the organization resources locally. Setting up a way to test changes locally, as opposed to a centrally managed CodePipeline, is somewhat more involved than creating separate Amazon S3 buckets. Because the state stored in Amazon S3 updates after every change we make to the organization, use it to ensure that the main pipeline will not skip a task because it was executed locally.

To review changes to the organization before applying them, use the org-formation create-change-set command to create a change set and execute-change-set to apply the changes after review:

\> org-formation create-change-set organization.yml --change-set-name my-change-set

\> org-formation execute-change-set my-change-set

Note that --change-set-name is optional when creating a change set. By default, a random identifier is used as the change set name.

Creating a new AWS account

We can add a new AWS account by adding a new OC::ORG::Account resource to the organization.yml file. As we don’t know the AccountId for the new account (i.e., the physical ID of an AWS account), use the RootEmail as an unique identifier for the AWS account. The console output will contain the AccountId after the account creation and we can add it to the organization.yml later, although doing so is not required. The AccountId will be stored in the state file in S3.

Adding a new AWS account to the organization file:

 MyNewAccount:
    Type: OC::ORG::Account
    Properties:
      AccountName: My New Account
      RootEmail: aws-accounts+new@myorg.com

Note that:

• AWS accounts that do not belong to an organizational unit are added to the organization root. We can add an account to an organizational unit by adding the account to the Accounts attribute using !Ref MyNewAccount. The Accounts attribute can be either an array or single !Ref.
• The root user for this newly created account will not have a password. To log in as root, we must reset the root password using the email address configured as RootEmail; thus, this email must be unique. AWS accepts email addresses that contain a + symbol and most mail providers allow a + to create another email address for the same mailbox. This can be useful if we want password recovery emails—and other emails that relate to our accounts—all sent to the same mailbox.

If we have additional steps that must be performed after creating an account—such as notifying another department by email, or adding the newly created account to list of accounts on our wiki—we can use Amazon EventBridge (or Amazon CloudWatch Events) to subscribe to AWS accounts being created by org-formation. The eventSource for these events is oc.org-formation. The event is AccountCreated.

Find an example of how to integrate a simple step function in GitHub.

Additional OC::ORG::Account attributes

In addition to AccountName, RootEmail, and AccountId, we can specify the following attributes:

• ServiceControlPolicies apply a list of service control policies to our AWS account.
• Tags add metadata to AWS accounts. Adding tags to our accounts is particularly useful because the value of these tags can be resolved using !GetAtt in org-formation Annotated CloudFormation templates. This way, the value of the tags can configure resources within these accounts.
• Alias creates an IAM alias associated with the account. This makes logging in to the account easier, as we can use the IAM alias instead of the 12-digit account ID.
• PasswordPolicy sets up a password policy for the account being created.
• SupportLevel sets the support level of the new account to enterprise if enterprise support is enabled in the main account.
• OrganizationAccessRoleName can be used to specify a specific IAM role name that will be used to access the AWS account. If the IAM role name is specified when the account is created, the IAM role will be created as part of the account creation process. Otherwise, the IAM role is assumed to be present in the target account and assumed to have the right permissions.

The following is an example of a fully configured AWS account:

MyNewAccount:
    Type: OC::ORG::Account
    Properties:
      AccountName: My New Account
      RootEmail: aws-accounts+new@myorg.com
      SupportLevel: enterprise
      Alias: org-newacc
      PasswordPolicy: !Ref PasswordPolicy
      Tags:
        Subdomain: newaccount.myorg.com
        BudgetThreshold: 100
        AccountOwnerEmail: olaf@myorg.com

  PasswordPolicy:
    Type: OC::ORG::PasswordPolicy
    Properties:
      MinimumPasswordLength: 12
      RequireLowercaseCharacters: true
      RequireNumbers: true
      RequireSymbols: true
      RequireUppercaseCharacters: true
      AllowUsersToChangePassword: true

Find additional information about other resource creations in the AWSOrganizationFormation documentation on GitHub.

Conclusion

In Part 1, we have covered the basics of how to initialize org-formation and start managing AWS Organizations resources using IaC. We also looked at creating and configuring a new AWS account. In Part 2, we will learn how to automate updates to an organization using an AWS CodePipeline.

• Part 1: Managing AWS Organizations resources using infrastructure as code
• Part 2: Integrating management of resources across accounts using task files
• Part 3: Deploying CloudFormation resources to multiple accounts using Organization Bindings
• org-formation on GitHub

The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.