AWS Cloud Operations & Migrations Blog
Using AWS Systems Manager Inventory with Tags
AWS Systems Manager Inventory now supports Tags. Tags enable you to categorize your AWS resources in several ways, for example, by purpose, function, owner, or environment. Consequently, when you use AWS Systems Manager Inventory to collect metadata from an instance, it also collects the tag information attached to the instance, making the tag information available as part of the inventory metadata of the instance. This allows customers to filter or query inventory metadata based on the instance categorization represented by tags, for example, you can query inventory only for instances in a production environment.
Further, when you use Resource Data Sync to sync inventory data to an Amazon S3 bucket, the tag data is also synchronized to the S3 bucket. This enables additional analytics and query capabilities on inventory data in the S3 bucket. When you use Amazon Athena and Amazon QuickSight, now tags can come in handy when you need to analyze or visualize the categorized inventory data.
In this blog post I’ll illustrate the new tag-based capabilities enabled by AWS Systems Manager Inventory, including the following:
- Filter or query inventory data based on tags attached to the instances
- Use tags to query inventory data from the Resource Data Sync Amazon S3 bucket
- Use tags to query inventory data using Amazon Athena and Amazon QuickSight
Prerequisites
a. Configure inventory collection on your managed instances*
This step is the starting point for configuring the inventory collection of your managed instances. You can configure inventory collection from the AWS Systems Manager console or by using the CreateAssociation CLI or API action. Inventory data can be collected by various target criteria — all managed instances, specified tag, or list of managed instances. Note that, when you collect inventory from an instance, tag information attached to the instance is collected automatically.
Additionally, to query inventory data aggregated across multiple AWS accounts and Regions, you can send inventory data from these Regions and accounts to a single Amazon S3 bucket using Resource Data Sync.
* A managed instance is an EC2 instance or on-premises instance that has AWS Systems Manager Agent (also called SSM Agent) installed and has an IAM role with an AmazonEC2RoleforSSM policy attached to it. For more details about managed instances, refer to the documentation.
b. Attach tags to your managed instances
If your instances are not tagged already, you can tag your instances in the EC2 console. Select Tags in the left navigation pane. You can attach a pre-defined tag or add a new tag if need be by choosing Manage Tags.
New and enhanced inventory capabilities
1: Filter or query inventory data based on tags attached to the instances.
Now that you have collected inventory data from your managed instances, you can leverage AWS Systems Manager Inventory filter / search capabilities. You can use the AWS Systems Manager console or the GetInventory and ListInventoryEntries CLI or API actions to query instances that meet your filter criteria. As of today, supported filters include Resource Groups, Tags, and Inventory Types. We’ll be focusing on filtering by tags in this blog post.
Let’s say we have a collection of managed EC2 instances. The following screenshot shows a list of managed instances in the EC2 console that is filtered by the required tag keys.
Open the AWS Systems Manager console and select Inventory from the left navigation pane. On the Filter by resource groups, tags or inventory types page, choose the filter box. A list of predefined filters drops down. Select the Tag attribute to filter on.
Selecting the Tag attribute presents you with a match condition of Equal. Select Equal and type in either the tag Key or the tag Key:Value.
Here is an example of filtering just by the tag Key.
Choose the Enter/Return key to list the managed instances that have a tag with Key as “Tier”.
Here is an example of filtering by the Key:Value.
Choose the Enter/Return key to list out the managed instances that have a tag with Key as “tier” and Value as “solo”.
The ability to filter instance inventory data based on the Tag Key or the Tag Key:Value helps filter the inventory data based on the environment being represented by the tag.
Note that the filter string for tag Key and Key:Value is case-insensitive.
2: Filter data based on Tags in combination with other attributes of the instance.
You can now use Tags to further filter results using other inventory types such as Service, Files etc. Let’s say, you have the following tags that help identify production and development instances.
Now, if you had to find production instances for which the Windows Firewall service is stopped or running, you could combine the AWS:Service filter with the Tag filter as shown in the following screenshot:
Now suppose, in one of the production instances the service had stopped running. The following screenshot shows an example of what you would see.
You can also use AWS CLI and API to filter the previous search. Here is an example showing the CLI command for the previous search:
As you can see, Tags are now part of the Inventory type and can be used for filtering inventory data in combination with other filter types.
3: Use Resource Data Sync to sync inventory data and tag information from the instance.
When you set up Resource Data Sync (refer to the steps here), tag data will automatically sync to the designated Amazon S3 bucket. In the S3 bucket, the path of the Tag data would look similar to the following, depending on your bucket layout:
S3 Bucket Name/[optional prefix]/AWS:Tag/accountid=123456789012/region=xx-xxxx-x/...
4: Use Amazon Athena to query instance inventory data based on the tag attached to the instance.
If you have configured Athena with the S3 bucket you used for Resource Data Sync, now you can run Athena queries based on tags. Here are some examples:
Example 1: The following Athena query shows how you can find all windows updates on production (e.g., tag:key=”environment” and tag:key:Value=”prod”) instances after a specific date:
Example 2: The following Athena query shows how you can find all executables installed on a development instance (e.g., tag:key=”environment” and tag:key:Value=”dev”). Note: I set up an AWS:File Inventory using following parameters:
select DISTINCT(aws_file.name) from aws_file inner join aws_tag on aws_file.resourceid=aws_tag.resourceid where aws_tag.value='Dev' AND aws_file.name like '%.exe';
5: Use Amazon QuickSight to build smart visualizations of inventory data based on the tag attached to the instance.
Using Amazon QuickSight, you can create data sets from any of the tables or join tables that you create in Athena to visualize your instance software inventory. A simple example visualization from the aws_tag table that follows provides you a quick way to identify any missing tags from your managed inventory estate.
Additional information on using Amazon QuickSight with Amazon Athena and AWS Systems Manager can be found in this helpful blog.
Tip: The crawler feature of the AWS Glue service along with its Data catalog component provides a quick and simple way to create the metadata tables from your inventory data in Amazon S3. It also provides an automated integration with Amazon Athena for immediate search and query.
Conclusion
In this blog post I focused on the key use-cases where Tag support in AWS Systems Manager Inventory can help enable enhanced query and analytics on inventory data. I started with filtering and querying inventory data based on tags. I then showed how you can leverage tags to perform advanced analytics and visualization on aggregated inventory data in Amazon S3 (synced using Resource Data Sync) using Amazon Athena and Amazon QuickSight.
About the Author
Imtiaz (Taz) Sayed is a Sr. Technical Account Manager and an engineer at heart. He loves working with customers and enabling them with solutions that allow them to do more with less.