AWS CloudFormation resource for Resource Data Sync in AWS Systems Manager

The Resource Data Sync capability in AWS Systems Manager lets you sync inventory data from your managed instances into an Amazon S3 bucket. Resource Data Sync then automatically updates the S3 bucket whenever new Inventory data is collected. You can also use Resource Data Sync to sync inventory data from multiple AWS accounts into a single Amazon S3 bucket, making the S3 bucket as inventory data lake for multiple AWS accounts. You can then use the data lake for advanced queries and analysis of inventory data across multiple accounts. For more information on Resource Data Sync Configuration, click here.

AWS CloudFormation simplifies provisioning and management on AWS infrastructure. You can create templates (JSON or YAML formatted files) for the service or application architectures you want and have AWS CloudFormation use those templates for quick and reliable provisioning of the services or applications (called “stacks”).

Recently AWS CloudFormation has added a resource for Resource Data Sync (AWS::SSM::ResourceDataSync). Using this resource, you can configure Resource Data Sync for one or more AWS accounts in a reliable and repeatable manner.

Previously, you had to use the AWS CLI or the AWS Systems Manager console each time you wanted to configure Resource Data Sync for an account. If you had to set up multiple Resource Data Sync for the same account (for example, where you want to sync inventory data to more than a single Amazon S3 bucket) or for for multiple AWS accounts, you had to remember previous configuration (such as S3 bucket details) and repeat all of the steps.

Now, with Resource Data Sync as a CloudFormation resource, you can manage Resource Data Sync configuration as code (via CloudFormation template) in the form of a simple YAML or JSON file. This means that you can check-in Resource Data Sync configuration template for version control and review it with team members before deploying to production. Later, you can reuse the template to configure/create Resource Data Sync in one or more accounts in an automated manner, without having to perform any manual actions.

Note that before you create a Resource Data Sync with CloudFormation, you need to meet the following prerequisites:

• S3 bucket where you want to sync your inventory data to, has been created.
• The S3 bucket has been configured with the correct bucket policy so that AWS Systems Manager can write to this bucket.
• If you want to encrypt the sync data with KMS Key, make sure you have a KMS key with the correct policy.

Once the prerequisites are met, add your configuration details in a template like the one below for creating Resource Data Sync with CloudFormation:

Description: "Basic Resource Data Sync Test"
Resources:
        BasicResourceDataSync: 
      	Type: "AWS::SSM::ResourceDataSync"
      	Properties:
          SyncName: "Test-Resource-Data-Sync"
          BucketName: "test-s3-bucket-resource-data-sync"
          BucketRegion: "us-east-1"
          SyncFormat: "JsonSerDe"
          BucketPrefix: "test"
          KMSKeyArn: "<provide your KMS Ker ARN>

Go to AWS CloudFormation console and create a stack called TestCFNStack with above template. Then validate that the stack TestCFNStack and the Resource Data Sync resource named BasicResourceDataSync have reached “CREATE_COMPLETE” state as shown below:

Once the stack has reached “CREATE_COMPLETE” state, it will create a Resource Data Sync in current region of the current AWS account, with the name Test-Resource-Data-Sync. You can validate it by going to Resource Data Sync in the AWS Systems Manager console as shown below:

Once the Resource Data Sync is created, it will start syncing basic details about your managed instances and the inventory, if configured, from your managed instances to the S3 bucket named Test-S3-Bucket in the us-east-1 Region, under the subdirectory sync-test with format as JsonSerDe. Note that the only valid format that can be used for Resource Data Sync is JsonSerDe. You can also use an AWS KMS key to encrypt the Resource Data Sync. You must specify a key in the same Region as the destination Amazon S3 bucket.

Resource Data Sync is an immutable configuration change. So whenever you want to update a CloudFormation stack that contains the Resource Data Sync resource, you have to use a different SyncName. This will create a Resource Data Sync with the updated template and then delete the old Resource Data Sync, thus keeping all the dependencies intact.

Conclusion

In this blog, I describe how to create a Resource Data Sync with an AWS CloudFormation template using AWS::SSM::ResourceDataSync, a newly added resource in the AWS CloudFormation. This new capability has made the configuration and maintenance of Resource Data Sync fast, reliable and repeatable.

About the Author

Utsa Bhattacharjya is Software Development Engineer at AWS Systems Manager. She is passionate about software development. Apart from work, she enjoys reading books and traveling.