Cost Control Blog Series #2: Automate Cost Control using AWS Service Catalog and AWS Budgets

This blog post is contributed by Pubali Sen, Senior Solutions Architect

In our “Good intentions don’t work, but cost control mechanisms do!” blog, we highlighted four cost control components, among which user permissions come first. The proper user permission sets up guardrails right at where actions take place. Customers let us know that they want native, automated spend management capability at the point of self-service resource provisioning. AWS Service Catalog allows you to pre-approve services for your users. With its integration with AWS Budgets, you can create and associate budgets with portfolios and products, and keep your developers informed the resource costs for them to run cost-aware workloads.

In this blog post, we will walk you through how you can set up a serverless automated workflow to govern the cost for your AWS Service Catalog portfolio.

About the workflow

We will build a serverless automated cost governance blueprint to automate the budget management for a given portfolio, and set up cost control guardrails for your non-production AWS environment. We will use the workflows in the blog “Enable self-service, secured data science using Amazon SageMaker notebooks and AWS Service Catalog” as an example and build a secured cost-governance framework around it.

Let’s say a developer has permissions to launch Amazon SageMaker at the beginning of the month, when the monthly budget is sufficient.  However, when the forecasted cost is at 60% of the monthly budget, the spend management automation will kick in.  The developer is still allowed to launch SageMaker, however, he/she is restricted to launch only smaller instance sizes and family.  For instance, he/she will lose the permission to launch new GPU instances and is restricted to smaller instance sizes and family for the rest of the month. When the forecasted cost reaches 95% of the allowed monthly budget, the spend management automation will prevent the developer from launching SageMaker.  This automation eliminates any possibility of exceeding the monthly budget and gives cloud admins the ability to keep the budget in check.

Solution Architecture

Below are the steps that show how this automated spend management works at a high level. The number of each step is labeled in the diagram above.

1. The budget for AWS Service Catalog Product “SageMaker Project” – is set up. With this feature, you can create and associate budgets with Service Catalog portfolios and products and track your spend.
2. The budget setup also includes settings to send a notification on an SNS topic when the usage is about to exceed the budgeted amount. For more information, see Creating an Amazon SNS Topic for Budget Notifications.
3. The master Lambda function receives the SNS notification.
4. It triggers execution of a Step Functions state machine with the parameters for completing the configured action.
5. The action Lambda function is triggered as a task in the state machine. The function interacts with IAM to effectively remove the user’s permissions to run the product catalog or spin new AWS SageMaker resources.

This decoupled modular design is scalable and can add new actions serially or in parallel with additional steps.

Implement the workflow

The CloudFormation stack includes creating the following resources:

• AWS Service Catalog Portfolio and Product
• Two AWS Lambda functions
• One Amazon SNS topic
• One Amazon IAM group
• One Amazon IAM user to test the solution
• Amazon IAM policies as needed
• One AWS Budget
• Associates the budget created with the AWS Service Catalog Product

To create the self-service cost governance solution stack, log in to your AWS console and go to CloudFormation.  Use the link below to launch your stack. It takes a few minutes to spin up the stack. You can monitor the progress in the CloudFormation console.

The AWS CloudFormation templates need the following inputs from you:

• Budget amount
• Budget comparison operator  (Let you set value points for comparison to ‘GREATER_THAN | LESS_THAN | EQUAL_TO’ . This Comparison can be a percentage or a value of your actual or forecasted AWS bill)
• Budget Notification type
• Tag information
• Environment name for this project

Figure 1: Launch the AWS CloudFormation Template

Figure 2: Provide the Parameter Inputs

When you see the CREATE_COMPLETE status for the stack you have created, choose Outputs. Copy the following Three values that you need later to verify the user permissions.

• UserName
• SignInURL
• Password

Figure 3: Outputs after resource creation

Verify the stack

Launch the product in the service catalog portfolio, and resources needed for Project Beta, and tag the resources needed for Project Beta with “Project, Beta”.

1. Browse to the SignInURL, and log in using the UserName and Password values copied on from the stack output.
2. In the AWS Service Catalog console, choose the portfolio created and launch the product.
3. It takes a few minutes to spin up the stack, and you can monitor the progress in the AWS Service Catalog console.

Verify the solution

First, identify the test IAM group that was created in the previous section. The group should have “projectBeta” in the name, prepended with the CloudFormation stack name and appended with an alphanumeric string. Verify whether the managed policy associated is: “Administrator”, which indicates that the users in this group have access to Service Catalog and can launch a product. Login via the user credentials and go to service catalog dashboard. You will be able to view the portfolio and the product and launch the product as well.

There are two ways of verification for this serverless automated spend control solution: simulating a notification, or waiting for a breach. Because it takes at least a few hours for the aggregate cost of the running resources to meet our budget thresholds, you can verify the solution by simulating the notification from Budgets.

Simulated notification

Log in to the SNS console (using your regular AWS credentials).

1. Publish a message on the SNS topic that has “budgetNotificationTopic” in the name. The complete name is appended by the CloudFormation stack identifier.
2. Copy the following text as the body of the notification: “This is a mock notification”.
3. Choose Publish.
4. Open the IAM console to verify that the policy for the test group has been switched to “SageMakerReadOnly”. This prevents the user from launching the product and alters his permission to service catalog.

Conclusions

We hope that you find this workflow helpful in governing and implementing the cost control mechanisms for your organization.  Learn more about AWS Cost Management and AWS Budgets and let us know what solutions you want us to cover next.