AWS Budgets Update – Track Cloud Costs and Usage

As Spider-Man and others before him have said, “with great power comes great responsibility.” In the on-demand, pay-as-you-go cloud world, this means that you need to be an informed, responsible consumer. In a corporate environment, this means that you need to pay attention to budgets and to spending, and to make sure that your actual spend is in line with your projections. With AWS in use across multiple projects and departments, tracking and forecasting becomes more involved.

Today we are making some important upgrades to the AWS Budgets feature (read New – AWS Budgets and Forecasts for background information). This feature is designed to be used by Finance Managers, Project Managers, and VP-level DevOps folks (please feel free to share this post with similarly-titled members of your organization if you are not directly responsible for your cloud budget).  You can use AWS Budgets to maintain a unified view of your costs and usage for specific categories that you define, and you can sign up for automated notifications that provide you with detailed status information (over or under budget) so that you can identify potential issues and take action to prevent undesired actual or forecasted overruns.

AWS Budgets Updates
You can create up to 20,000 budgets per payer account. In order to allow you to stay on top of your spending in environments where costs and resource consumption are changing frequently, the budgets are evaluated four times per day. Notifications are delivered via email or programmatically (an Amazon Simple Notification Service (Amazon SNS) message), so that you can take manual, semi-automated, or fully automated corrective action.  This gives you the power to address all of the following situations, along with others that may arise withing your organization:

VP – Optimize your overall cloud spend, with budgets for each business unit and for the company as a whole, tracking spending by region and other dimensions and comparing actual usage against budgets.

Project Manager – Manage costs within your department, watching multiple services, tags, and regions. Alert stakeholders when thresholds have been breached, and ask them to take action. When necessary, give resource budgets to individual team members to encourage adoption and experimentation.

Finance Manager – Analyze historical costs for your organization and use your insight into future plans to develop suitable budgets. Examine costs across the entire company, or on a per-account, per-service, business unit, or project team level.

Creating a Budget
Let’s create a budget or two!

Start by opening up Billing and Cost Management:

And then click on Budgets:

If you are new to AWS Budgets, you may have to wait up to 24 hours after clicking Create budget before you can proceed to the next step. During this time, we’ll prepare the first set of Detailed Billing Reports for your account.

Click on Create budget, decide whether you want the budget to be based on costs or usage, and give your budget a name. Then select Monthly, Quarterly, or Annual. I’ll go for a cost-based ($1000) monthly budget named MainBudget to get started:

By not checking any of the options next to Include costs related to, my budget will apply to my entire account. Checking a box opens the door to all sorts of additional options that give you a lot of flexibility. Here’s how I could create a budget for usage of EC2 instances where the Owner tag is set to jbarr:

I could be even more specific, and choose to set a very modest budget for usage that is on Non-Reserved instances. This would be a great way to make sure that I am making good use of any Reserved Instances that my organization owns.

The next step is to set up email or programmatic notifications:

The programmatic notification option can be used in many different ways. I could create a new web app with a fixed budget, and then invoke a AWS Lambda function if costs are approaching the budgeted amount. The app could take corrective action to ensure that the budget is not exceeded. For example, it could temporarily disable some of the more computationally intensive features, or it could switch over to a statically hosted alternative site.

With everything set up as desired I simply click on Create. My budget is visible right away (I clicked on the triangle in order to display the details before I took this screen shot):

As you can can see, I have already overspent my $1000 budget, with a forecast of almost $5,600 for the month. Given that we are a frugal company (read our leadership principles to learn more), I really need to see what’s going on and clean up some of my extra instances! Because I had opted for email notification, I received the following message not too long after I created my budget:

Suppose that my data transfer budget is separate from my compute budget, and that I am allowed to transfer up to 100 GB of data out of S3 every month, regardless of the cost at the time. I can create a budget that looks like this:

And I can see at a glance that I am in no danger of exceeding my data transfer budget:

I can also download the on-screen information in CSV form for further inspection or as input to another part of my budgeting process:

As you can see, this new feature gives you the power to set up very detailed budgets. Although I have introduced this feature using the AWS Management Console, you can also set up budgets by making calls to the new Budget API or by using the AWS Command Line Interface (AWS CLI). This API includes functions like CreateBudget, DescribeBudget, and UpdateBudget that you can use from within your own applications.

Available Now
This new feature is available now and you can start using it today! You can create two budgets per account at no charge; additional budgets cost $0.02 per day (again, you can have up to 20,000 budgets per account).

To learn more, read Managing Your Costs with Budgets.

— Jeff;