AWS Cloud Operations Blog

Centralized software package distribution across multiple regions and accounts in an AWS Organization using AWS Systems Manager Distributor

Security remains a top priority for most organizations, and, in order to stay secure and compliant, they leverage agent-based vulnerability management tools, such as CrowdStrike, TrendMicro, and Tenable. AWS Systems Manager Distributor automates the process of packaging and publishing software to managed Windows and Linux instances across the cloud landscape, as well as to on-premises servers, via a single simplified interface. Customers can leverage Distributor to package and manage security and other monitoring agents, and then distribute them to cloud and on-premises instances as well.

Most customers have well-established landing zones built via AWS Organizations, with several AWS accounts spread across multiple regions. They want to centrally automate software package management and distribution across this large landscape. This post will demonstrate how you can build a solution that automates your software package distribution across all of your member accounts spread across regions within an AWS Organization via AWS Systems Manager. The key features leveraged are Automation, Distributor and State Manager.

Note: This solution works in all GovCloud and commercial regions regions except China.

Prerequisites

The following prerequisites need to be completed to deploy this solution.

  1. Prepare your .zip files of installable assets, with one .zip file per operating system platform. At least one .zip file is required to create a package.
  2. Create a JSON manifest. The manifest includes pointers to your package code files.
  3. When you have your required code files added to a folder or directory, and the manifest is populated with correct values, then upload your package to an Amazon Simple Storage Service (S3) bucket. These have been described in the first 3 steps of Create a package (advanced).
  4. This solution uses the management account within AWS Organizations, but you can also designate an account (delegated administrator) to manage this on behalf of the organization. If you intend to use a delegated account then you will need to register it as delegated administrator for CloudFormation stack set operations as described in Register a delegated administrator.

Note: To ensure optimal package management, we recommend only having the relevant files for a package uploaded in its own prefix within the S3 bucket that is utilized solely for this purpose.

Solution overview

Distributor, a capability of AWS Systems Manager, helps you package your software within a region in an account. The Centralized Software Distributor (CSD) solution described in this post enhances your ability to manage and distribute software packages across member accounts that are spread across regions in an AWS Organization.

Figure 1 shows the solution architecture. In addition to Distributor, the solution utilizes the following Systems Manager capabilities.

  • Automation to orchestrate the workflow.
  • State Manager to invoke the Automation document on a user-defined frequency.

Admin must upload installable assets (zip and manifest files) as a prerequisite. In step 1, the State Manager invokes the CSD_AddSoftwarePackageToDistributor automation, which in step 2 adds a package to AWS Systems Manager Distributor. In step 3, the Distributor retrieves the package contents (installable assets and manifest files) from the S3 bucket. In step 4, Automation invokes the CSD-DistributeSoftwarePackage document. In step 5, Automation sends the command to Distributor along with the target instances. In step 6, Distributor deploys the software package to those instances.

Figure 1: Solution architecture

Walkthrough

To deploy the solution, download this CloudFormation template and deploy it in your organization’s management account.

The template deploys the following resources:

  1. Automation documents
    • CSD-AddSoftwarePackageToDistributor which contains the logic to package the installable software assets and add it to Distributor.
    • CSD-DistributeSoftwarePackage which contains the logic to distribute the software package to every target instance in the member accounts.
  2. IAM roles

Provide the following inputs for the template:

  • IsDelegatedAdminstrator: Specify if the solution will use a delegated administrator account within the Organization to manage the software packages. CloudFormation StackSet IAM roles should be provisioned.
  • DelegatedAdminAccountId: (Optional) Delegated administrator account ID.
  • ManagementAccountId: (Required) AWS Organization’s Management account ID.
  • DeploymentTargets: Specify AWS account IDs and/or the organizational unit IDs within AWS Organization whose accounts have the target instances (e.g., ou-abcd-1qwert43, 123456789123) for distribution
  • TargetKey: Specify which instances have to be targeted for this solution. Allowed values – ParameterValues, ResourceGroup or begin with tag:, AWS::EC2::Instance, InstanceIds (default), instanceids. Refer to Target for more details.
  • TargetValues: Specify the target key values specified above. Default is *, which targets every instance. Refer to Target for more details.

Visit our GitHub repository to access the individual resource templates.

Invoking the solution using a State Manager association

A State Manager association is a configuration assigned to your managed instances. The configuration defines the state you want to maintain on your instances.

Update the highlighted parameters, and then execute this command in the management account of your organization to create a new association.

  • TARGET_ACCOUNTS: AWS organizational unit IDs within AWS Organization whose accounts have the target instances (e.g., [“ou-abcd-1qwert43”, “ou-pwyt-1q345ft”]) for distribution.
  • TARGET_REGIONS: All AWS Regions (e.g., [“us-east-1″,”ap-southeast-2”]) where the target instances are running.
  • MAX_CONCURRENCY and MAX_ERRORS: Specify these values based on the number of accounts and error thresholds described in TargetLocation in the AWS Systems Manager API Reference.
  • ACTION: Specify whether or not to install or uninstall the package.
  • INSTALLATION_TYPE: Specify the installation type. Uninstall and reinstall: The application is taken offline until the reinstallation process completes. In-place update: The application is available while new or updated files are added to the installation.
  • PACKAGE_NAME: The package to install/uninstall.
  • VERSION: The package version to install or uninstall. The system will only attempt to uninstall the version currently installed. If no version of the package is installed, then the system returns an error.
  • ADDITIONAL_ARGUMENTS: The additional parameters to provide to your install, uninstall, or update scripts.
  • S3_BUCKET_NAME: Specify the S3 bucket name where the package has been deployed to. This bucket should only consist of the packages and its manifest file.
  • S3_BUCKET_PREFIX: Specify the S3 prefix where the package assets are stored.
  • AUTOMATION_ROLE_ARN: ARN of the CSD-SystemsManager-AutomationAdministrationRole.
aws ssm create-association \
    --association-name "csd-association" \
    --name "CSD-AddSoftwarePackageToDistributor" \
    --parameters '{"targetAccounts":["TARGET_ACCOUNTS"],"targetRegions":["TARGET_REGIONS"],"maxConcurrency":["MAX_CONCURRENCY"],"maxErrors":["MAX_ERRORS"],"action":["ACTION"],"installationType":["INSTALLATION_TYPE"],"name":["PACKAGE_NAME"],"version":["VERSION"],"additionalArguments":["{ADDITIONAL_ARGUMENTS}"],"AutomationAssumeRole":["AUTOMATION_ROLE_ARN"],"bucketName":["S3_BUCKET_NAME"],"bucketPrefix":["S3_BUCKET_PREFIX"]}' \
    --no-apply-only-at-cron-interval

This command will invoke the system to run it once immediately after it is created. To update it to run on a scheduled basis using --schedule-expression, see create-association in the AWS CLI Command Reference.

Validating the execution ran successfully

After the association has triggered the automation, open the Systems Manager console, and from the left navigation pane choose Automation. In Automation executions, you should see the most recent execution of both CSD-AddSoftwarePackageToDistributor and CSD-DistributeSoftwarePackage, as shown in Figure 2.

On the Executions tab, the Automation executions are displayed in a table organized by execution ID, document name, status, start time, end time, and executed by.

Figure 2: Automation executions (management account)

Click each execution to validate if they were completed successfully.

Depending on the number of regions, accounts, and instances that you execute this solution against, a successful run of the execution looks like the following in the CSD-DistributeSoftwarePackage execution details:

On the Execution detail page for the CSD-DistributeSoftwarePackage, there are sections for execution description, outputs, status, and executed steps.

Figure 3: CSD-DistributeSoftwarePackage execution detail (management account)

To ensure that the package was deployed to the targeted member account instances, navigate to the Systems Manager dashboard and select Run Command in the left pane. Under Command history, you should be able to see every invocation and their status.

In the Command history tab for Run Command, there is a list of every command executed, along with their ID, status, requested date, document name, targets, error, and completed.

Figure 4: Run command history (member account)

Click any Command ID, and then select the instance ID for the command output. Check the Ouput section for both steps. On a successful completion, you can expect a similar output as shown in Figure 5.

In the Outputs section, there are two parts each providing a command description and status for both the steps, along with the specific output saying that the package was installed successfully.

Figure 5: Run Command output (member account)

Adding a package version

To add a package version:

  1. Upload the modified package along with the new manifest file to the same S3 bucket under the designated prefix where the previous package assets where stored.
  2. Update the current association with the new version using update-association. Ensure that you provide the same version that was specified in the manifest.json to the association.

Adding new accounts, OUs, and Regions

The solution will automatically deploy to any new accounts that you provision under the OUs specified when you originally deployed the CloudFormation template. The addition of new accounts or OUs will require updates to both CloudFormation and State Manager association as described below. The addition of new regions will only require updating the association.

  • CloudFormation :
    1. In the CloudFormation console, choose the original template you deployed, and then choose Update.
    2. Leave the Use the current template option selected.
    3. Under Automation details, update the STACKSET_TARGETS
  • State Manager Association:

Update the association using the instructions provided in Editing and creating a new version of an association with the new accounts, OUs, and/or Regions.

Conclusion

This post demonstrated how you can leverage AWS Systems Manager Distributor, as well as some of its other capabilities, to build a centralized software package distribution solution across all of your AWS accounts and Regions within an AWS Organization. With a consistent and reliable package management and distribution framework in place, you can now maintain your security and compliance posture across not only AWS, but also your on-premises infrastructure.

About the author

Praveen Bhat

Praveen Bhat

Praveen Bhat is an Enterprise Solutions Architect with several years of experience in the technology industry. By using his passion to bridge the gap between technology and business, Praveen has helped banking, insurance, manufacturing, government, wagering, and media organizations achieve their business objectives.

❌