Cost Allocation Blog Series #2: AWS-Generated vs. User-Defined Cost Allocation Tag

This blog post is contributed by Harsh Patel, Associate Cloud Developer

In my last blog “Cost Allocation Basics That You Need to Know”, I shared three foundational elements 1) account structure, 2) tagging, and 3) reporting on shared costs, you need to establish before you can implement a successful cost allocation model that meets your business needs. Tagging, as a great way to define the ownership and usage purpose of your resources in the format of metadata, allows you to quickly manage and filter your resources and trace costs back to the right entities. In this blog post, Harsh Patel, our Associate Cloud Developer will share his advice on how you use the two types of cost allocation tags: AWS-generated and User-defined.

AWS provides you the flexibility to initiate resources and services at any time – you pay only for what you use. As your usage of AWS grows, cost awareness and accountability should become your organizational priority.  You may have received a bill in the past that was higher than what you expected, or spent more time than you wanted to find out what these costs came from. With a growing number of users building applications on AWS, tracking your costs to the right entity can be difficult. This is why a clear resource tagging strategy becomes important. Tags can help you manage resources which would otherwise have names that are not intuitive or less meaningful to you.  Tags can also improve transparency and map costs to the specific products or applications. This association of costs via a tag is known as cost allocation tags, once activated, a cost allocation tag organizes your resource costs and enables you to track those costs at a detailed level via cost allocation report or AWS Cost Explorer.

AWS provides two types of cost allocation tags: User-defined and AWS generated.

• AWS-generated Tags: These are “createdBy” tags that are automatically defined, created and managed by AWS.
• User-defined AWS Tags: These tags are the ones that you need to define, create and apply manually to each service. The tag key can include anything that you’d like to track, ranging from projects to cost centers.

We will compare both options and analyze which one works better for you. Before we get started, ensure you have either a master account or a single account that isn’t a member of an organization. It’s important to note that only master or single accounts have access to the Cost Allocation Tags manager in the Billing and Cost Management console.

AWS-Generated Cost Allocation Tag

It sometimes can be a tedious task to remember and apply the cost allocation tag consistently to each and every service that gets launched. For example, having to apply a tag to every EC2 image is manual and time-consuming. To some organizations, it could be hard to implement a tagging strategy that ensures each and every service gets tagged. This is why we launched the AWS generated cost allocation tag. A predefined tag (key = createBy) that automatically creates and applies to the appropriate services as they get launched. With this AWS- generated cost allocation tag, you do not need to worry about applying the cost allocation tag to every service you are launching. The key benefit of using AWS-generated tag is that once it is activated, all of the member accounts will have it enabled by default. Thus, from an organization perspective, you no longer need to worry about which accounts have enabled the cost allocation tag and which ones have not.

You can activate the AWS generated tag in the Billing and Cost Management console with four easy steps.

1.  Sign in to the AWS Management Console and open the Billing and Cost Management console
2.  In the navigation pane, choose Cost Allocation Tags
3.  Under User-Defined Cost Allocation Tags, choose the createdBy tag (note: even though this is in the User-Defined Cost Allocation Tag section the createdBy tag enables the AWS-Generated Cost Allocation Tag)
4. Choose Activate

The createdBy tag uses the following key-value definition:

It can take up to 24 hours for tags to activate. After the tag is activated, AWS starts applying the tag to resources that are created after the AWS generated tags was enabled. Please note that the createdBy tag does not count towards your tags per resource limit, where each AWS resource can have a maximum of 50 user created tags.

Things to Consider When Using: AWS-Generated Cost Allocation Tag

There’s one major catch — a limited number of AWS services are supported by this generated tag. You can find the list of services that support the AWS generated tag at launch in this user guide.  As you can see, a few services such as Lambda, RDS and SNS are not yet on the list. The other thing to note is that once we have it activated it applies to all of the services (listed in the table) during launch time; you don’t have the option to opt out of a specific service. For example, you may not want this tag applied to EC2 images, yet want it applied to all Cloud Formation stacks, you can’t specify what service can/cannot inherit the tag. By default, all services listed in the table will inherit the tag. Due to these two limitations you may find it helpful to take a hybrid approach and also enable the user-defined tags, this way you can ensure the remaining AWS services contain the cost allocation tag and appear in AWS Cost Explorer or other reports.

User-Defined Cost Allocation Tag

User-defined tags are tags that you define, create, and apply to resources manually. This option gives you the flexibility and customization in picking what tag we want to define as user-defined and allows you to apply those tags to specific services. Since this is a traditional approach, we first have to decide a cost tagging model. For example, let’s say we determine that the key “cost-center” will be our cost allocation tag. We then have to apply this user-defined tag to all our AWS services otherwise they will not appear in AWS Cost Explorer or any of our reports.

After you create and apply the tag we’ll go ahead and activate with four easy steps.

1.  Sign in to the AWS Management Console and open the Billing and Cost Management console
2.  In the navigation pane, choose Cost Allocation Tags
3.  Under User-Defined Cost Allocation Tags, choose the tags that you created and applied, in our case we decided to proceed with “cost-center”.
4. Choose Activate

After activation please note that it will take 24 hours to appear in the Billing and Cost Management console.

Things to Consider When Using: User-Defined Cost Allocation Tag

Applying the user-defined tags can be a manual and time-consuming process. For example, if one of your account users forgets to add the “cost-center” tag to an EC2 instance, that service will then miss this particular cost allocation tag and will not appear in AWS Cost Explorer or any of our reports, if you filter by the “cost center” tag.  Similarly, if the user misspells the tag, you will have the same outcome.

Cost Allocation Report

Once you have activated the tags, AWS generates a cost allocation report as a comma-separated value (CSV file) with your usage and cost grouped by your active tags. You can enable this report by following this user-guide. This report will live in an S3 bucket that you specify. One takeaway is that in the cost allocation report, the user-defined tag keys have the prefix “user”, such as user:Cost Center and user:Application, AWS generated tag keys have the prefix “aws”, thus if you enable the AWS-Generated tag you’ll find the aws:createdBy column. This enables you to organize the charges for resources and get a different perspective on what/how projects are spending on AWS. For example, if you enable “Owner”, “Stack”, “Cost Center” and “Application” as the user-defined tag, you can track the total cost of a single application that runs on those resources. The following screenshot shows a partial report with columns for each tag.

Conclusion

In this blog post we have shown how you can activate AWS-generated and user-defined cost allocation tags. We discussed the differences between the two options and the limitations that may apply. Lastly we took a look at a sample cost allocation report to help you manage service costs as your usage of AWS continues to grow. For more information about activating the AWS generated tags, see Activating the AWS-Generated Cost Allocation Tags. For more information about applying and activating user-defined tags, see User-Defined Cost Allocation Tags.