How to establish and drive a forecasting culture

In part two of our forecasting blog series, we’re going to explore how you can establish a cost forecasting culture in your organization.

As budgeting season kicks off (or maybe you’re already in it), typical forecasting challenges become magnified when projecting future cloud costs. Without a robust forecasting governance mechanism, the flexibility of decentralized cloud infrastructure ownership can result in siloed forecasts and limited understanding, especially as thousands of workloads are rolled up for central reporting.

A key stakeholder in the forecasting process is the team that helps forecast and govern finances in the cloud. In your organization it might be called any number of names such as: Cloud Financial Management, Cloud Operations, Cloud Center of Excellence, or FinOps team. For the purpose of this blog, we’re going to refer to this team as FinOps.

As a central partner, this team is the go-to for both Finance and Engineering, and is in a unique position to create a forecasting mindset and culture.

Following the steps below can help a FinOps team institute forecasting norms and ultimately increase cloud business value.

Establish a common cost language

Will all stakeholders give the same number if asked the question, ‘What are we spending on cloud?’ If the answer is ‘no’, establishing a common cost language is crucial. A common cost language determines how cloud costs are grouped (account, workload, specific tags, etc.), and which pricing model will be used in cross-functional discussions (on-demand, blended, etc.). Stakeholders should align on both grouping and the pricing model prior to beginning the forecasting season.

For example, imagine Finance works on a forecast model using on-demand pricing, expecting to apply Savings Plan (SP)/Reserved Instance (RI) benefits after discussion with engineering. At the same time, Engineering provides a model using a blended rate with SP/RI benefits already included. The result is that the SP/RI benefits will eventually be double-counted. As with many of life’s problems, upfront communication is key, and can prevent negative downstream impacts of incorrect forecasts due to lacking transparency.

A cost language doesn’t have to solely focus on the pricing model – unit cost modeling can provide another forecasting avenue. Forecasting volume/instance growth, and incorporating current and future unit cost projections, can help bridge the gap between Engineering and Finance, with each stakeholder owning portions of the model.

Establishing guidelines around cost forecasting (on-demand, blended rate, centrally-provided rates, etc.) will prevent guesswork and limit issues with reconciling actuals vs. forecast in future periods due to misaligned rates.

Establish timelines and (over)communicate

Forecasting timelines are likely set by Central Finance/Planning, so awareness of major deliverables is key. Sharing the forecasting cycle with all levels, from engineers and developers to senior leadership, will facilitate alignment and encourage prioritization of budgeting exercises.

Use both regular (monthly finance/business reviews) and budgeting cycle kick-off meetings to share timelines and expectations. Examples include:

• Key dates (initial kickoff, first pass of forecast due, senior leadership review, final forecast locked)
• Common cost language (on-demand, blended rate, centrally-provided rates, etc.)
• SP/RI coverage expectations (flat to current levels, increasing at x% based on assumed growth)
• Level of forecast detail (complete forecast refresh, only significant changes, etc.)
• Forecast periods (rest of year, next 18 months, etc.)
• Enterprise initiative impacts (governance, resiliency requirements, optimization targets, etc.)

Be prescriptive from the get-go. Ambiguity leads to misaligned messaging as individual forecasts are rolled up to senior levels.

Establish central data/tooling expectations. Focus on the material.

With multiple accounts and workloads, detailed forecasting at the granular level is neither realistic nor particularly necessary. Developing a methodical approach and establishing forecasting expectations will allow teams to focus on the material workloads while still creating reasonable projections for the remaining spend.

In forecasting, cost visibility is paramount, and tooling such as the Cost Intelligence Dashboard can help. Here we lay out recommendations of ways your cloud operations or FinOps team can facilitate forecasting along three distinct spend buckets.

1. New workloads, or existing workloads with large expected future growth

With a lack of historical data to reference, new workloads can be the most challenging to forecast, as well as the ones most likely to drive large forecast variances as usage increases. As Engineering scopes out a new workload, the AWS Pricing Calculator should be used to forecast steady-state cost, both in prod and non-prod environments. With the steady-state architecture and cost, Engineering and Finance can model out the projected monthly cost ramp. Additionally, Finance can apply any necessary central financial treatments, such as SP/RI benefits. The end result is a monthly view of a new workload, with well-documented assumptions, to incorporate into the forecast.

2. The top existing workloads

Analyzing your existing workloads will likely result in spend roughly following the Pareto Principle (the 80/20 rule), where a large percentage of spend is driven by relatively few accounts/workloads. These workloads will most likely drive any meaningful variances of actuals to forecast, and should be the focus of detailed forecasting efforts.

Developing central tooling/templates can support forecasting of these top workloads, both in terms of dollars and, more importantly, the assumptions behind those dollars. Templates showing historical workload spend by environment and top services, with mechanisms to project future dollars by either percentage growth or absolute dollars, will enable conversations between Engineering and Finance as stakeholders review templates. Additionally, completed templates will assist with variance analysis in future months, as the assumptions should be well documented.

3. All other workloads

Remaining spend will likely be made up of a large number of relatively small-dollar workloads. Attempting a detailed forecast for smaller dollars may not be the most efficient use of time. However, you may not want to use a blanket growth assumption. The FinOps team can support the smaller workload forecasting effort by creating a data package of historical spend trends by workload. Such a data package can then be used to establish agreed-upon growth assumptions, taking into account business priorities.

Ultimately, the tools are of secondary importance to the process. A regular communication cadence will drive alignment and establish stakeholder trust. The FinOps team’s goal should be to facilitate forecasting ownership among all stakeholders.

Iterate, Iterate, Iterate

Forecasting isn’t an endpoint; it’s a journey. A forecast is a point-in-time estimate of future spend based on current state and best-guess assumptions. Well-documented assumptions then provide a base for explanation as cost profiles change and actuals vary from forecasts.

Importantly, variance from forecasts should be blameless. Rather, documenting assumptions will enable you to explain variances, and adjust your outlook accordingly.

Final thoughts: Forecasting is a partnership

Forecasting cannot happen in a silo. Finance needs Engineering support to understand future usage. Engineering/Developers must communicate with Finance to ensure future expectations are incorporated within the official budget. The FinOps team acts as a facilitator to deliver the correct information and the proper detail for all stakeholders.

Forecasting season magnifies the importance of cloud education among all stakeholders. Resources such as AWS Well-Architected, and AWS Twitch TV (including the The AWS Keys to Optimization show) can increase cloud financial acumen among all parties. Finance can drive deeper conversations with Engineering as cloud knowledge grows. Conversely, the more that Engineering understands the financial process, the better the alignment of expectations and messaging.

Forecasting is challenging enough without having to deal with process ambiguity. By establishing a robust forecasting mechanism, the FinOps team can facilitate more productive conversations and a sense of ownership among all parties.

Stay tuned for part three of our forecasting series, where we will do a deep-dive on driver-based forecasting.

📖 ICYMI: Part 1 of the forecasting blog series – How to improve your cloud cost forecasting