Using Kubernetes Labels to Split and Track Application Costs on Amazon EKS

We’re excited to announce support for Kubernetes labels in split cost allocation data for Amazon Elastic Kubernetes Service (EKS). With this launch, you can now import Kubernetes labels as user-defined cost allocation tags into split cost allocation data, allowing you to attribute the cost of your applications running on an Amazon EKS cluster using these labels in AWS Cost and Usage Reports (CUR). This enables you to allocate your Kubernetes costs based on your specific business requirements and organizational design. For details please visit using Kubernetes labels for cost allocation in Amazon EKS.

Split cost allocation data provides granular cost and usage information for pod-level resources on Amazon Elastic Container Service (ECS) and Amazon EKS. For workloads or applications running on Amazon EKS, you can use predefined metadata like aws:eks:cluster-name, aws:eks:namespace, etc. in split cost allocation data to create cost allocation reports. However, you may want additional flexibility to attribute costs — for example, allocating the usage of shared resources to specific teams, business units, or cost centers. It is a common practice to use Kubernetes labels on pods to track ownership of applications and would like to use these metadata along with system labels like cluster name and namespace to allocate their costs. Below you will learn how to leverage Kubernetes labels to further define and allocate granular pod level costs in Amazon EKS.

Example use-cases

Let’s consider a scenario where an organization operates a shared Amazon EKS cluster running multiple applications. The infrastructure hosts Amazon EMR on Amazon EKS for big data processing alongside traditional web applications, with distinct ownership divided between two departments, Research and IT. Although this architecture optimizes resource utilization, the multi-tenant nature of this setup introduces complexity in cost attribution. The FinOps team wants to implement an accurate cost allocation model to determine application-specific expenses and facilitate appropriate departmental billing. To address the cost allocation challenge, the organization implemented a labeling and tagging strategy, using Kubernetes labels for Kubernetes(k8s) resources, and AWS Tags for AWS resources. The two use cases below are examples of how to improve cost visibility in a shared multi-tenant Kubernetes environment.

Use-case 1 – Cost visibility for Spark applications in multi-tenant environment

When running big data applications like Apache Spark applications on Amazon EMR on Amazon EKS, pods are automatically labeled using K8s labels that identify the Spark app id and Spark app name, as well as unique identifiers related to Amazon EMR on Amazon EKS. In Figure 1 below, a label spark-app-name = spark_job_a will be created and assigned to pods upon creation. These labels can now be imported and presented as cost allocation tags upon activation. Using these labels, you can use AWS Cost and Usage Reports to allocate cost to a specific Spark application, using the Spark app id or name. Organizations can also track total Spark application costs by using the unique Amazon EMR on Amazon EKS identifier that appears both as a pod label and in the AWS Cost and Usage Report line items, enabling cost allocation across both compute resources and Amazon EMR service charges.

Figure 1. Using labels to classify applications in Amazon EKS cluster

Use-case 2 – Application total cost of ownership

The applications running on the shared Amazon EKS cluster also use other AWS resources, such as Amazon Relational Database Service (RDS) instances, Amazon Simple Storage Service (S3) buckets or Amazon OpenSearch Service domains. The applications owners would like to know the total cost of ownership (TCO) for their applications, including the K8s resources (pods) and AWS resources (Amazon RDS, Amazon S3, and OpenSearch). With consistent labeling and tagging across the K8s and AWS resources, the applications owners can allocate the total cost of their applications across all resources. An an example, in Figure 2 below, an application named web_app consists of multiple pods, as well as the AWS resources mentioned above. The application pods can be labeled with label key app having label value web_app, and the AWS resources can be tagged with tag key app having tag value web_app. Notice both the K8s label and the AWS tag have the same key and value. Once the labeling is done, the application owner can use AWS Cost and Usage Reports to sum the split cost of the pods and the cost of the AWS resources, by the app cost allocation tag, which represents both the K8s label and the AWS tag. As a result, the application owner allocated the total cost of ownership for the application. This can be done for multiple applications running on the same Amazon EKS cluster, and on multiple Amazon EKS clusters.

Figure 2. Using labels to calculate TCO of applications in an Amazon EKS cluster

Split cost allocation data pulls in the Kubernetes labels as cost allocation tags, and propagates this metadata into the CUR. Leveraging this detailed cost information, customers can now write custom queries or utilize business intelligence (BI) tools to generate consolidated reports that aggregate all related costs into a single view. This approach enables the organization to calculate the cost of serving each application within their shared infrastructure, providing visibility into unit economics, and financial performance across their cloud environment.

Get started with Split Cost Allocation Data and Kubernetes labels

Prerequisites:

• AWS account
• Amazon EKS cluster for which you want to track split cost allocation data. This can be an existing cluster, or you can create a new one. For more information, see Create an Amazon EKS cluster in the Amazon EKS User Guide. The labels for cost allocation can be added to Kubernetes pods using kubectl , K8s manifests or programmatically through CI/CD automation.

Step 1: Setup Split Cost Allocation Data for Amazon EKS

• From the management account you need to opt-in “Split cost allocation data” from your AWS Billing and Cost Management Console page. Open the Billing and Cost Management console at https://console.aws.amazon.com/costmanagement/
• Navigate to Cost Management Preferences and in the General section select Amazon Elastic Kubernetes Service as in Figure 3 below.
  • You have three options to get pod-level metrics: Resource requests, Amazon Managed Service for Prometheus, or Amazon CloudWatch Container Insights. See the documentation on how to setup these options.

Figure 3. Split cost allocation data in Cost Management preferences

Step 2 : Activate cost allocation tags

The imported labels will show as cost allocation tags for the pod resource type. Cost allocation tags can be activated from Management accounts only. See Figure 4 below.

• Go to the AWS Management Console and open the AWS Billing and Cost Management console at https://console.aws.amazon.com/costmanagement/.
  In the navigation pane, choose Cost allocation tags.
• Select the tags that you want to activate, for example app.kubernetes.io/name.
• Step 4: Choose Activate.

Figure 4. Activating Cost Allocation Tags

Step 3: Setup CUR for Split cost allocation data

• This can be accomplished from a management or linked account
• Navigate to AWS Billing and Cost Management console and click on Data Exports page. From the list select the CUR you want to edit.
• To create a new CUR, click on Create
• Check the Split cost allocation data option
  • Both Legacy CUR and CUR 2.0 are supported. We recommend including resource IDs and setting an hourly report to get the most granular data. See Figure 5 below

Figure 5. Split Cost Allocation Data in Data Exports for CUR 2.0

For more details on the opt in process visit Enabling split cost allocation data.

After split cost allocation data is enabled, split cost allocation data will scan for all your Kubernetes pods in Amazon EKS clusters across your Consolidated Billing family and ingest Kubernetes attributes such as namespace, node, cluster, CPU, and memory requests for the pods. Pod-level Kubernetes labels are also imported and can be activated as user-defined cost allocation tags.

For a detailed understanding of how split cost allocation works, see the blog Improve cost visibility of Amazon EKS with AWS Split Cost Allocation Data.

As split cost allocation data calculates Kubernetes pod-level metrics, new columns will appear in your CUR reports. It can take up to 24 hours for the data to be visible in AWS CUR. For each Kubernetes label, a corresponding new column is generated, and the data associated with the pod is populated within this newly created column. For instance, a new column named resourceTags/user:app.kubernetes.io/name will be created for the label “app.kubernetes.io/name” that was added in the previous steps. See Figure 6 below.

Figure 6. Sample Cost and Usage Report columns

Managing Kubernetes labels and cost allocation tags

Once split cost allocation data is operational, you can add, delete, and update the Kubernetes labels. Please see the documentation for detailed steps. To understand the best practices to create Kubernetes labels for cost allocation, see Understanding split cost allocation data.

Conclusion

The ability to visualize application costs running on Amazon EKS using Kubernetes labels enables organizations to gain valuable insights into their spending patterns. This flexibility helps teams optimize efficiency and align cost allocation with their organizational structure. By leveraging these label-based insights, businesses can make more informed decisions about resource utilization and implement more accurate cost attribution across different teams, projects, and departments. There is no additional charge for enabling split cost allocation data. However, standard S3 storage fees will apply for storing the incremental column data added to CUR. Please see this page for estimating the number of line items in CUR. This feature is available in AWS Regions where split cost allocation data for Amazon EKS is available. You can also use the Containers Cost Allocation dashboard to visualize your application costs running on Amazon EKS in Amazon QuickSight and CUR query library to query your Amazon EKS costs using Amazon Athena.