Selecting a unit metric to support your business

Voiced by Amazon Polly

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details.

Having discussed what unit metrics are, real-life use cases, and the benefits they offer, we’ll share how you can create a good unit metric.

So, how do you pick a good denominator for our unit metric formula?

A good demand driver has a statistically significant relationship to cloud usage. We are not trying to portray ourselves as statisticians here. You can look up how the Pearson correlation coefficient is constructed and how it works if you want to dive deeper. Keep it simple, buy an actuary or data scientist their favorite form of caffeine and ask for a favor.

In short, the correlation coefficient (R²) between two variables measures the strength of the relationship between them on a scale from -1.0 to +1.0 (total negative correlation to total positive correlation).  The closer the R² is to -1 or +1, the better our demand driver can act as a predictor of spend and resource consumption. With a high level of correlation, either positive or negative, the better our unit metric will be at representing marginal cost and marginal consumption.

When and if possible, pick demand drivers that are easily relatable to business or end user functions they are to represent. In a perfect world, there would be one unit metric for each product or service being offered. Often times there is the need for more than one unit metric depending on how customers interact with a given service or product.

Single Unit Metric example:

Your company provides a SaaS offering that scans objects in an Amazon S3 bucket to look for anything that could be malicious or represent a security threat.  Customers call a public facing API endpoint and provide a pointer to the object to be scanned.  The SaaS offering then returns a code indicating the status of the scan’s findings.

The demand driver in this case is the number of API endpoint return codes over a given period of time.  The more scans that are performed, the more AWS resources are used.  Why not count the number of API calls?  A call to the SaaS API doesn’t create customer value.  The results of the scan returned by the API is what delivers value to the customer.  For that reason, we recommend we count API responses.

Multiple Unit Metrics example:

You operate a marketplace to enable ham radio operators, something near and dear to my (Mike’s) heart, to list used equipment they have for sale in hopes that someone will buy it so you’ll have money and space to buy new radio gear.  If you’re a ham radio operator, that last part makes perfect sense.

Here you will have multiple demand drivers due to the nature of how users interact with the system.  By breaking down the customer experience, we can identify the major business functions that are to be supported:

1. Listings – in a marketplace, there is a need to have something to sell.  Our sellers will upload one or more pictures of what they want to sell along with a description of the goods for sale and the asking price.  The most likely financial unit metric will be cost per listing.  One of the more important engineering unit metrics for this part of the system will be GB of Amazon S3 storage, or GB of Amazon CloudFront Data Transfer Out per listing.
2. Searches – buyers need a way to find that much needed “new to them” piece of equipment.  Our buyers will use our search engine to look through the inventory of goods for sale.  Cost per search and resources used per search are going to be the finance unit metrics for this portion of the infrastructure.  For the engineering unit metrics, depending on how we plan to break down the resources used, they could be Amazon EC2 compute time per search, units of Amazon OpenSearch Service (successor to Amazon Elasticsearch Service), number of Amazon CloudFront HTTP/HTTPS Requests, and/or units of Amazon ElastiCache used per search, etc.
3. Purchases – when a buyer decides to make that piece of equipment their own, they will engage in the commerce itself and make the buy.  Cost per payment processed would be a great candidate for a financial unit metric.  For the engineering unit metrics, let’s say we store all order information in Amazon DynamoDB, then units of read request per payment will be a good unit metric.

Adding together the cost per listing + cost per search + cost to process a purchase provides a means of reporting the expense of operating your marketplace’s infrastructure on AWS.

Hopefully, this post provided a high level overview of how to identify good demand drivers and then considerations on how to apply them based upon the way your products and services are constructed.

Next time, we’ll share some of the bumps to watch out for when building unit metrics with the hopes that they can be avoided in your journey.