Risk and Plans: More Constraints Requires More Agility

I am often asked a question along these lines: Mark, I get that Agile and DevOps approaches favor learning and adjusting over rigidly following a plan. But our budgeting process locks in our spending (and therefore our annual plan) in advance, and our investment management process necessarily locks in spending levels and schedules for each project. So how can we still be Agile when our spending is set in stone ahead of time?

Let me make the question harder for myself by adding a few other special cases. Public companies must give the markets some guidance on their projected earnings per share, and once they do, they are pretty much locked in to delivering. And in a scenario I often encounter—contractors have often locked in delivery schedules and other constraints by contract.

In all of these cases, the commitments that are locked in ahead of time limit agility—that is, limit the company’s ability to change course as it learns and as the business environment changes. As I have often in these blog posts, I’m going to frame this as a question about risk, although there are plenty of other ways to think about it. Locking in spending or delivery dates in advance is not in itself a problem: it simply adds a risk which must be mitigated.

Agility Within a Budget

We’ll look at the budget scenario in particular, but I think the logic applies to any of the other “non-agile” constraints. Let’s say that the company, in its annual budgeting process, locks in spending for a particular category at the beginning of the year. The way budget processes typically work, they are probably pre-determining their spending even further in advance than a year, since the budget process itself takes time. In the government, it is actually more than a two-year effort. The implication is that the company knows that far in advance what it will want to spend, or at least is willing to manage to that amount even if it turns out to be less than optimal.

As the budget year plays out, there will be some difference between the budgeted and ideal spends—in other words, between the amount in the budget and the amount that the company would ideally be spending as the year progresses and circumstances change. If the company has done a perfect job predicting its year, then the difference will be zero. We know that this is not realistic. What we often don’t acknowledge is just how unrealistic it is. The digital age is one of rapid change, uncertainty, and complexity. And as these characteristics continue to increase, so too will the difference between ideal spend and budgeted spend.

For example, let’s say that as of midyear, the company is on target to spend its budget for computer infrastructure exactly. Now it realizes that spending an additional $1,000 on infrastructure before the year ends would allow it to serve an additional 100 customers and increase revenues by $1,000,000. If the company is rigidly managing to budget, it will forego the incremental $1,000,000 in revenue, which is bad. Or let’s say that by spending an additional $1,000 beyond IT’s budget, it can deploy code that will reduce costs in a line of business by $1,000,000. Or perhaps someone in the company has a brilliant idea for a new product or feature that will drive significant revenues. Again, the budget limit may constrain the company from implementing the idea or force a delay until the next budget year (and there is a time value of money). These are just a few examples of what I mean by the difference between ideal and budgeted spending.

Are these likely or unlikely events? Somewhere in between. But with the amount of change and complexity we see today, it is very likely that something will cause a discrepancy between ideal and budget. Note that the difference could be in the other direction as well: let’s say that a vendor unexpectedly reduces its prices. In that case the ideal spend would be lessthan the budgeted spend.

Constraints Add Risk (in an uncertain environment)

Because of the uncertainty involved, I like to think of this discrepancy as a risk rather than a certain cost. When the company locks in its IT spending, it is taking a risk that the ideal will be different from the amount locked in. There is nothing wrong with taking risk, per se. In the examples above, the risk is that it will leave money on the table, even though it could be increasing revenues or decreasing costs—that is, it is risking an opportunity cost. But the risks could be more serious and direct. Perhaps a competitor makes a sudden competitive move and the company is reluctant to respond because doing so would put it over budget. In that case, it might be a risk of losing the entire company.

Plans, at least when the company intends to follow them, impose risks, because the future (over the time that the plan is in effect) is uncertain. Again, that is not necessarily a bad thing. But it leads to two questions:

• Does the advantage of making and following the plan outweigh the risks?
• How can the company mitigate the risk it is accepting by making the plan?

We consider budgets to be highly valuable because they are control mechanisms—they make sure that someone in the organization doesn’t cause an unwelcome surprise by letting their spending get out of control. But there may be alternative ways to accomplish the same thing with less risk. I have written about using staged or metered investments as a way to control project-based spending, essentially re-making the decision of how much to spend as the company sees the results of earlier phases of the project. There may be other control mechanisms that can be used that have the advantages both of curbing runaway spending and permitting agility. There is a Beyond Budgeting movement that promotes a number of ideas along these lines—ideas that have been used effectively at large enterprises worldwide.

But let’s assume that the company is unwilling to forego annual budgets—as most are. Indeed, I wouldn’t presume to suggest that they do, because the budget process has so many side effects that each company must consider the pros and cons for itself.

Mitigating the Budget’s Risk

Which brings us to question two, how to mitigate the risk of deciding on a budget or a project plan too early, before the actual circumstances of the year unfold. And risk mitigation will take the form of finding ways to introduce flexibility even while capping spending at the budgeted level. It turns out that this is one area where DevOps and Agile approaches really shine. With the old Waterfall approach, exceeding budget was always a serious possibility, because you committed to a set of requirements in advance and then kept spending until you completed them. With an Agile approach, you can commit to a level of spending (or team capacity), and then keep re-prioritizing the work items to get the most value until the budget is spent, at which time the work stops.

You can also manage the risk of having chosen a fixed spending level by building IT architectures (probably microservices) that provide as much flexibility as possible, or by training your employees to be generalists (T-shaped DevOps team members). You will then be able to reduce the risk of your fixed budget level by taking advantage of the flexibilities available to you within that budget level.

Another possibility is to accept budget overages—but only when they are correlated with increases in revenue. That is, you can try to make more of your costs variable rather than fixed, so they will grow with revenues as part of the cost of goods sold. Here, the cloud provides much of the necessary magic, as infrastructure and service usage will vary with the volume of business, which will in turn depend on revenue. In other blog posts, we have written about how using serverless techniques in the cloud can make this connection transparent and manageable.

In a way, Agile approaches are a way of living within budget, whereas waterfall is a way of refusing to do so. Waterfall says, essentially, no matter how much it winds up costing us, we are going to complete this set of requirements, and we will just blame our projections or performance if we do go over budget. Its assumption is that planned cost will equal actual cost, so there is no problem. But that is a bad assumption in times of uncertainty. Agile, on the other hand, says that if we find we are running out of budget, then we will do fewer things from our wish list, and since we keep adjusting priorities, we know we will have done the most important and high-return tasks before we run out of money.

So when people ask me how they can be Agile even in the face of fixed constraints imposed in advance, especially constraints on budget, I suggest that they reverse the question. Given those constraints, how can they not be Agile? They must use all of the Agile technique available to them to reduce the risk the company is taking on by choosing to downplay uncertainty in formulating those plans in advance. There is no reason why you can’t be Agile with budget constraints, but there is plenty of reason why you can’t not be Agile.

Yes, it seems strange to think of the budget as a risk, but there you have it (you are welcome to argue that not making a budget also has risks).

Mark