What is a unit metric?

Voiced by Amazon Polly

From time to time, customers will ask me questions like, “my AWS bills are going up and I don’t know whether that’s a good thing or a bad thing?” Did the bill go up because you’re operating more workloads in the cloud or is it because you’re using AWS inefficiently? Perhaps a little of both are going on. Regardless of the reason, the most often encountered interpretation is that more spend is bad! Period. End of story.

When your AWS invoice is viewed without the appropriate context, it is hard to tell if an increase in spend is a result of delivering more value from your use of the cloud or if it is due to inefficient and wasteful cloud resource consumption. Being able to tell the difference is important. You’re now likely asking, well, how do I tell the difference? Simply stated, with unit metrics.

Some of these explanations will be using some fancy terminology especially if you don’t have a cost accounting background. Fear not, you will have a good understanding of unit metrics by the time you finish reading this post. A unit metric represents the rate by which a demand driver causes change to an entity. In this case, that entity is AWS resource consumption, which is directly related to your AWS bill. A unit metric can be expressed as a rate of spend or as a rate of resource consumption. Both representations will have value for your organization depending on the audience. A unit metric is a Key Performance Indicator (KPI) that is composed of a numerator which quantifies an amount of spend or resource consumption and a denominator that quantifies our demand driver in units of demand.

figure 1

The objective of a unit metric is to present incremental cost or incremental consumption in terms of a unit of the demand driver.  Now that we’ve used this term “demand driver” several times, let’s define it.  In the world of Cloud Financial Management (CFM), a demand driver is a factor that is correlated to AWS spend or AWS resource consumption. The quantity of AWS resources consumed and the cost of utilizing those resources are directly impacted by increases or decreases in the demand driver.

Finance and accounting teams would refer to the cost unit metric as “marginal cost”. A manufacturing professional may be familiar with the term “marginal consumption” or the amount of inputs (made or purchased) one would find on a bill of material. In the cloud, we’re going to use both of the marginal cost and marginal consumption versions of a unit metric as KPIs, so we can tell a complete story around whether or not our cloud spend is delivering more business value.

Identify a demand driver candidate in terms of the business activity that maps directly to your organization.  If you operate an airline, maybe you’re going to use Available Seat Miles (ASM) which is an industry standard, if you’re operating a rideshare, using the number of trips taken seems like a great place to start.  This makes it more relatable to the value your products and services are delivering to your customers.  For example:

The more you have of the demand driver, the more AWS resources you will utilize, and in turn the greater your AWS spend.  The converse is true as well.  When there is a decrease in the demand driver, AWS resource usage decreases and AWS spend should scale down to align with the demand.

Tying this back to where we started – as I review this month’s AWS bill, how do I know if I’m moving in the right direction or the wrong direction with respect to cost?  And more importantly, what is driving that change?

For our example, monthly AWS cost is represented by the blue vertical bars that are increasing over time.  By itself, all you can really say is my AWS costs are growing.  The default answer is more AWS expense is a bad thing.  Without the context a unit metric provides, on its face, this seems like a reasonable answer.

figure 2

The context provided by the unit cost allows us to make a data-driven determination to assess if the increase in spend is a sign of success or a sign that something needs to be investigated.  A declining unit cost implies they are decreasing due to optimizations and improvements in your system’s architecture or operations, transactions are increasing due to volume growth, or a combination of both.  If the opposite is true and unit costs are increasing, then it is a sign that something needs to be investigated. There are a few cases where increased unit costs are okay, but for the most part, increases in the cost of delivering value to your customers are not a good sign.

Examples of where an increase in marginal costs is okay or are at least justifiable:

figure 3

• Deploying new code to fix a bug. Prior to the fix, the true cost of operations were understated since the product was not working correctly.
• Implementing  a new regulatory requirement or changes to an existing one. These are table stakes and are the cost of doing business. There is no getting around it.
• Deploying code that lays the groundwork for future enhancements to your products and services which will be followed by new revenue or a decrease in churn once the enhancements are released. The change in revenue or churn are measurable and can help determine the level of market adoption or the efficacy of efforts to reduce customer turnover.

A common use of unit metrics is to help explain why that gold line in the figure 2 is moving up or down.  With respect to CFM, unit metrics are a data driven way to assess the effectiveness of your governance and Cost Optimization (CO) efforts.  When planning for work to be slotted in a product’s backlog, you need an estimate of the cost or an estimate of the change to the existing cost as a part of the specifications.  This serves a couple of purposes.  First, it forces your architects and engineers to understand not only the technical aspects of the AWS services they are using, it also compels them to understand the cost levers for these services.  Secondly, it creates a reference point to compare expected costs with actual costs.  The difference between expected and actual costs is known as a variance.  Diving deep into understanding what is causing a variance will make your architects and engineers better at what they do.  In the pay-for-what-you-need for- only-as-long-as-you-need-it paradigm of cloud computing, having a mastery of the cost drivers of the services you use becomes just as important as having a technical mastery of them.

There are a few aspects of unit metrics I’d like to touch upon that can enable our IT organization to move in the right direction.

Unit metrics are an effective tool to help your organization implement:

• Forecasting – with projections provided by Sales and Marketing combined with the incremental unit costs for those products and services, you can now forecast the impact to the cost of operations.  Expected AWS resource consumption can be forecast as well.   This will prove to be helpful to assess if the additional workload causes your system to approach or exceed its operational limits.
• Chargebacks and Showbacks – cost accountants and analysts can treat the incremental unit cost as the standard cost for purposes of attributing expense to the proper ledger accounts.  Whether the costs are booked to a product’s cost center (chargeback) or if they remain within the IT organization (showback), being able to accurately quantify the cost of operating a product or service is powerful.
• Gross Margin Analysis – by tracking changes to the incremental unit cost after code releases, the financial impact of the release can be measured.  In a continuous release environment, you are going to focus more on cost trends, but for most everyone else, the impact of discreet changes to architecture and IT operations can be measured.  Breaking down unit cost to show each AWS Service’s contribution to cost will prove to be a great mechanism to identify and prioritize direct future efforts to reduce cost to get that bang for the buck.
• Organizational Alignment – well-functioning unit metrics provide a common point of reference for Engineering, Sales, Marketing, Pricing, and Product teams to ensure gross margins are being preserved as they relate to IT operations.

In coming installments of this series, we will discuss how to identify good unit metric candidates, bumps along to road that can be avoided though good planning and execution, and how unit metrics can be used to create cross functional alignment within your business.