How to improve your cloud cost forecasting

It’s that time of year again, when companies are starting their 2023 planning. Since technology usage is often an organization’s highest expenditure after personnel costs, effectively forecasting cloud spend is vital to planning, negotiating, and achieving sustainable economies of scale as you grow and mature your business on the cloud. So, what can you can do to more accurately predict future cloud costs? In particular, how can you forecast your AWS spend for the next month, quarter, or year?

To help you improve your cloud cost forecasting, we’re developing this four-part blog series so you can:

• Implement best practices to improve financial predictability
• Establish a cost forecasting culture in your organization
• Understand and build driver-based forecasts that take into account changes to what drives your business
• Use AWS tooling to get started with building and sharing your forecasts

Let’s kick off the first blog in the series by deep diving into the best practices you can implement to improve financial predictability:

1. Increase cross-functional collaboration
2. Perform driver-based forecasting
3. Establish governance and accountability

Increase financial visibility through cross-functional collaboration

You’ll want to begin by establishing a steering committee. Invite engineering, product, finance, and sales and marketing teams. Getting a diverse range of key cross-functional stakeholders is important, since this committee will be in charge of implementing measures and guardrails that ensure there’s improvement in financial predictability for investors and shareholders. This is key. Visibility into how much you’re currently spending, another component in successful demand planning, is vital.

Teams can use AWS tools such as Cost Explorer, Billing Conductor, Cost Intelligence Dashboards, and Cost Categories to visualize and gain deeper understanding of their spend, and the key drivers that influence spend outcomes. For example, Cost Intelligence Dashboards (in QuickSight) lets you slice and dice spending in multiple views including budget allocation, service, and tags.

A bi-weekly cadence, or at least a monthly cadence, between finance and engineering/product teams is an opportunity for cross-functional knowledge sharing to help educate one another, discuss root causes of potential cost variances, adjust forecasts, and maintain business alignment. Then, the output from these meetings can be shared with the steering committee on a monthly basis.

Sample output from AWS Budgets below shows that the product owner identified the root causes of cost variance: Compute (EC2) costs increased due to development delays and unexpected customer demand from a promotion. As a result, the Steering Committee reallocated budget from an un-performing product to the registration product, and increased the new Q4-22 budget to $70/mo to support ongoing customer promotions.

Plan for variable costs through driver-based forecasting

There are two primary types of forecasting approaches: trend-based and driver-based. Trend-based forecasting is based on historical spending patterns and predictive extrapolation of historical time-series data. This type of forecasting works well when you’re looking  6- to 12-months out, but once you get beyond the one-year mark, this forecasting is less certain. It’s also limited because it doesn’t factor in business changes such as new product launches or demos, product promotions, or pilot programs.

On the other hand, driver-based forecasting is ideal for planning variable costs since it is based on expected changes to demand drivers. Other benefits for driver-based forecasting include improvements in forecast accuracy, collaboration, and accountability.

These are the four primary demand driver categories:

• Look at your internal drivers, such as new product launches, new environments demo, pre-production, load testing, and changes to existing products; this can also include new features, new service adoption, re-architecture, or modernization.
• Next, factor in external drivers, such as new users, sales events, promotions, free trials, and seasonal use (i.e., Prime Day, holidays, Cyber Monday, etc.).
• Add in strategic drivers, such as regional or global market expansion, mergers and acquisitions, and divestiture.
• Finally, be sure to consider reverse demand drivers, such as customer churn, optimization, retiring workloads, or retiring environments.

As we mentioned earlier, we’ll be taking a closer look at building these driver-based forecasts, including a few specific examples, in our third blog of this series (coming soon!)

Establish financial accountability through strong governance

If this is your first attempt at cloud cost forecasting, start with a line of business or a particular product line. Assign 1 full-time employee (FTE) to create visibility reports and an initial planning tool template.

Understanding each stakeholder’s responsibilities is vital to ensuring proper governance and accountability.

The steering committee will lead quarterly cross-functional planning sessions. Finance is responsible for budget allocations on a quarterly basis, based on a bottoms-up demand plan. Sales teams offer sales plan inputs, marketing sheds light on marketing plans, demos, and new product/feature inputs, and product owners share their forecast accuracy to plan, make adjustments, and/or seek additional funding.

The best practice is for the product owners to assign cost accountabilities to each engineer or operator on a two-pizza team. What’s a two-pizza team? In the early days of Amazon, Jeff Bezos instituted a rule: that every internal team must be small enough to be fed with 2 pizzas. Thus, everyone on the team understands the profitability goals and the budget, and has visibility into the actual spend. New team members should be trained on the process through self-paced learning and by shadowing weekly meetings before generating their budget. Mature teams will include cost optimization stories in their backlog to address continuous improvement opportunities.

If the product owner misses their forecast, they must justify the root cause of the variance and document mitigation steps. This is a blameless exercise. At first, forecast accuracy will be low, but over time, teams should strive for >95% accuracy.

Mature teams should go one step further by incorporating checkbook functionality into the process for those teams that meet forecast goals. Teams can use their checkbook savings earned for additional training, trading with other departments, or innovation work. You can also gamify success by sharing best practices or sending out reports that highlight top performers, which earns bragging rights.

Start small. Begin operating and capture learnings before scaling the process to the remaining product lines and product owners.

Conclusion

Product teams are now empowered to create annual, quarterly, monthly, or even daily budgets depending on business needs. These reports give product teams the ability to spot anomalies early and take timely action to prevent cost or usage overage, or inefficient utilization or resource coverage of your Reserved Instances and Savings Plans.

Thanks to tools like AWS Budgets, which lets you set custom budgets, alerts, and triggered actions related to exceeding or falling below desired thresholds, you can build a decentralized cloud spend forecast. It’s easier than ever to collaborate with multidisciplinary teams across your organization to increase visibility, ownership, accountability, and predictability. With cross-functional teamwork, comprehensive driver-based forecasting, and strong governance, you can achieve better outcomes, improved forecast accuracy, and expanded margins, which all businesses strive to achieve.

Check in next week for our second blog in the series, where we’ll discuss how you can build a forecasting culture in your organization, establishing it as an ongoing practice. Because the more often you forecast, the more accurate you’ll be over time.