AWS Cloud Financial Management
Automating custom rates at scale: an Amazon case study with AWS Billing Conductor
While many see AWS and Amazon as one single entity from the outside, Amazon is actually a customer of AWS. Amazon has multiple business units like fulfillment centers, devices, stores, media & entertainment, and advertising that run workloads on AWS. These business units have unique needs and utilize custom internal rates to view their AWS cost.
In the following blog post, we discuss how we used AWS Billing Conductor to build a custom solution for Amazon to enable them to view their AWS cost at internal rates in AWS Cost Explorer and AWS Cost and Usage Report (CUR).
AWS Billing Conductor
AWS users often need to apply custom rates for internal charge-back or cost allocation models. By default, native AWS cost management tools like Cost Explorer and CUR do not provide functionality to customize rates. Amazon needs to view net AWS cost and track return on investments for their AWS workloads.
Amazon chose to explore AWS Billing Conductor (ABC), an AWS customizable billing service that allows you to customize your billing data to match your desired showback or chargeback business logic.

Figure 1. Product diagram of AWS Billing Conductor
A challenge that became apparent from the beginning was processing the large number of possible SKUs. With over two hundred AWS services and with each service possibly containing hundreds of usage types, the number of custom rates needed quickly added up. Adding these custom rates into ABC as pricing rules manually in the AWS console can take some time. Amazon needed a solution that could create custom rate SKUs and programmatically upload them as pricing rules results to ABC in bulk.
AWS Architecture
Amazon developed an AWS-based solution to streamline the loading of pricing plans, pricing rules, and billing groups into AWS Billing Conductor at scale. The architecture leverages several AWS services for a comprehensive approach:
- Amazon S3: Utilized as the primary data storage solution. Amazon S3 housed the rate sheets for Amazon, detailing their specific AWS usage types and associated costs. This setup facilitated centralized and secure access to critical pricing data.
- AWS Glue: Leveraged for its powerful ETL (Extract, Transform, Load) capabilities, AWS Glue refined the pricing data stored in S3. It parsed and removed irrelevant information streamlining the data preparation phase.
- Amazon Athena: Athena‘s interactive query service enabled the application to retrieve specific pricing information from the processed data stored in S3 and join them to create a single data set, which would be loaded into AWS Billing Conductor.
- AWS Billing Conductor: This service was pivotal in managing the billing framework. It was used to configure billing groups, pricing plans, and intricate pricing rules. A pro-forma Cost & Usage Report (CUR) was generated for each billing group, enabling detailed and customized billing insights tailored to the requirements.
- AWS Fargate: Hosts a custom Node JS application which served as an interface for managing AWS Billing Conductor pricing rules at scale (this can be replaced by the compute and language of your choice)
Each service contributed uniquely to the architecture, ensuring a seamless flow from data storage and preparation to analysis and billing management, ultimately providing a robust and tailored billing solution for Amazon.

Figure 2 – Architecture Diagram of the Proof of Concepts the Team Built
Management for Pricing Rules
When building the pricing rules, we encountered a challenge where Amazon’s custom rates did not align directly with those presented in the AWS public pricing API. This discrepancy necessitated a data cleanup to ensure smooth operation in downstream processes.
We significantly improved performance by grouping common rates that applied across usage types in order to reduce the total number of pricing rules. We then applied this pricing rule as a Global Scope within ABC which resulted in less pricing rules being handled by Athena and the Fargate container. This also reduced the need for excessive API calls, improving our processing efficiency by 35%. Additionally, we included rate-limiting mechanisms to ensure the system’s scalability and reliability without compromising performance.
Results
Once the PoC was completed, Amazon launched the Fargate app in their accounts and after a 24-hour period, they were able to generate a pro-forma CUR. After performing validation by comparing it to a internal financial tool, our team confirmed the viability of the both the automated pricing rule creation process as well as the accuracy of the ABC based cost date.
In total, thousands of rules were programmatically created in ABC, across a subset of product families and it was observed that the pro-forma CUR successfully replicated the existing internal cost allocation system. While there were some small discrepancies between the actual AWS monthly bill and the pro-forma CUR totals which were caused by the rounding of certain percentage discounts, these were easy fixes which were insignificant in the overall success of the PoC.
Conclusion
Like many enterprise cloud users, Amazon uses an internal charge-back mechanism that requires rate customization that are not by default represented in AWS billing and cost management services like Cost Explorer and the CUR. With the help of AWS Billing Conductor, Amazon was able view their AWS cost in the console in a way that accurately reflected their net AWS spend. With this solution, other AWS customers who need to apply custom rates can look to utilize AWS Billing Conductor for their AWS financial management needs to better understand their spend and budget accordingly.