How to Manage Costs Effectively
This article is part of a technical content series crafted by AWS Startup Solutions Architects to help guide early stage startups in setting the foundations needed to start building quickly and easily. The series offers a high-level overview of the technical decisions startup founders need to make when getting off the ground, along with which AWS services are best suited to address those decisions.
When your company is first starting out, all of the focus is on getting to market quickly. It’s not uncommon that a choice or two gets made that sacrifices a bit of cost efficiency for the sake of moving faster. However, as your product or service hits the market and your business starts to scale, staying on top of your costs should become a priority. The last thing you want is to see all of that hard-earned customer acquisition get eaten up by unnecessary operating expenses! In this article, we give you the top 5 best practices for understanding and staying on top of your AWS spend.
Enabling AWS Organizations
Up until now, you’ve likely been building your product or service in a single AWS account. This works well for self-learning, prototyping, or academic purposes, and it provides the most direct way to monitor and report on costs for your resources, such as Amazon EC2 instances, Amazon DynamoDB tables, Amazon Kinesis Streams, and so on. For a maturing startup, however, there are a number of benefits with our first best practice, which is to enable AWS Organizations. AWS Organizations is a service to create and manage accounts, place them in a hierarchy, and apply top-down policies that set the boundaries of what can be done within the accounts under that policy. These policies become a key component to controlling your costs as you scale.
We recommend you do this by first creating a second, brand new AWS account the same way you created your original one. Once created, enable AWS Organizations on that account to make it your new organization master. This master account is used specifically for collecting, monitoring, and reporting on costs, and your child accounts are where you will host your actual resources for development, production, security, and so on. You can then invite your original account to join your new Organization. Creating this separation and introducing additional accounts as your business grows helps keep your environments organized, your costs transparent, and your security easier to manage.
Leveraging the AWS Cost Management Center
The second practice is to get familiar with the AWS Cost Management center, which is where you find Cost Explorer. Here, all of your costs across your AWS accounts can be sliced and diced, drilled down into, and visualized from as granular as hourly to up to a year or more. You can also group your data by attributes such as Service, Region, Usage Type, and others to understand your cost and usage drivers. For example, you may want to find out how much your data transfer charges were between Availability Zones in us-west-2. Using Cost Explorer, set the Region to us-west-2, the Service to EC2, and the Usage Type Group to Inter AZ data transfer.
Back in the main Cost Management dashboard, you’ll find AWS also forecasts your spend for the remainder of the current month. AWS also looks for patterns in your usage to highlight trends and anomalies that might be worth digging further into. If you’ve not already done so, we highly encourage you to take a look at the Cost Management center today and see what insights it might show you.
Between Cost Explorer and the other dashboards, AWS provides a number of default dimensions across which you can analyze your spend. These dimensions are predominantly based on attributes about the infrastructure itself. They help you discover which service is generating which cost, but you may also want to align costs to dimensions that are important to your business. For example, you may want to know the cost of resources that are provisioned for a specific customer or the cost of specific sub-systems of your overall architecture, such as a group of microservices or a data pipeline.
Enable and Apply Tag Policies
These insights are enabled by our third best practice, which is to apply tags to your AWS resources. The easiest way to get started with tagging is to first define a Tag Policy for your Organization. Tag Policies let you specify which tags you wish to require across your accounts. Optionally, you can also specify their allowed values. You can then see which resources are non-compliant or configure the policy to enforce mandatory compliance. Common tags our customers use include ones such as Cost Center, Customer, Application, or Project. Find these and more ideas in our Tagging Strategies page.
If you’re not ready to enable AWS Organizations and want to start enforcing a tagging policy, you can also achieve this using a combination of IAM Policy Conditions and monitoring with AWS Config.
Applying tags themselves can be done in a number of ways. The most common is to simply define them as part of your Infrastructure as Code templates, such as with AWS CloudFormation or another tool of your choice. You can also automate applying tags through our Tagging API. Whichever option you use, we suggest you develop a tagging practice early, as it helps to reduce the overall effort of getting one in place.
Once done, specify which tags you want to activate for cost reporting purposes. This is done under the Cost Allocation Tags section of your Account Settings. Note that it can take up to 24 hours for these tags to show in both Cost Explorer and your Cost and Usage Report.
It’s one thing to regularly check Cost Explorer to understand your current and forecasted spend, but it’s even better if you can be proactively alerted when your spend looks like it’s going to exceed your desired threshold. Our next practice achieves just that using AWS Budgets.
Set Up Alerts with AWS Budgets
AWS Budgets provides a few different kinds of metrics to monitor, including Costs, Usage, Reservations, and Savings Plans, but the one to start with is a Cost Budget.
When setting up your Cost Budget, you can pick any of the AWS-provided dimensions or any custom dimensions you created through tagging. You can create a Budget that alerts when the spend for a specific service or resources that share a specific tag exceed either a set dollar amount or a percentage of your target spend. You can also configure multiple alerts on your budget, like having a first alert sent when you reach 50% of your spend threshold and a second alert at 80%.
Cost Influence Analysis
The final best practice is to take a look at the pricing for each service you use, understand the factors that contribute to its cost, what discount models are available, and see how to optimize your architecture around those attributes.
For example, DynamoDB is very popular amongst our startups due to its low overhead and high performance as a NoSQL database. Taking a look at the pricing page below shows you that there are two cost models: one for on-demand capacity and another for provisioned. Each page then details how various attributes, such as the number of requests and the size of a record, impact your cost.
In the case of reading from DynamoDB, we see that each read consists of one or more read request units. Each read request unit can fetch a max of 4KB of data. Additionally, we see that an eventually-consistent read only requires half of a read request unit, a strongly-consistent read requires one read request unit, and a transactional read requires two read request units. Thus, designing your application to use eventually-consistent reads while keeping individual records under 4KB allows you to save, at minimum, 4 times the cost over using transactions.
There are opportunities to minimize your costs in nearly every AWS service. By creating an Organizations master account, familiarizing yourself with Cost Explorer, creating and enforcing a tagging policy, leveraging Budgets, and understanding what attributes influence costs for the services you use and designing your application to minimize them, you’ll have no problem controlling and optimizing your costs as you scale.
Have fun, and build on!