Optimizing costs on Amazon DocumentDB using event-driven architecture and the AWS EventBridge Terraform module

A primary reason companies move their workloads to AWS is because of cost. With AWS, cloud migration and application modernization plans are based on your business needs and not agreements or licensing. You can acquire technology on an as-needed basis, only paying for the resources you use. You can build modern, scalable applications on AWS to transform your organization, all while optimizing costs.

In this post, we discuss how to optimize costs on Amazon DocumentDB (with MongoDB compatibility) using an event-driven architecture with AWS Lambda, Amazon EventBridge, and the AWS EventBridge Terraform module by automating the starting and stopping of clusters around business hours.

Cost-optimization

Amazon DocumentDB recently announced I/O-Optimized pricing, a new storage configuration that provides improved price-performance and predictable pricing for customers with I/O-intensive applications. If you’re running an Amazon DocumentDB cluster with Standard storage configuration, you can run the Amazon DocumentDB Deployment Scanner to estimate possible cost-optimization and utilization by moving to the I/O-Optimized storage configuration.

Also available is document compression for your Amazon DocumentDB 5.0 instance-based clusters. Through use of the LZ4 compression algorithm, compressed documents are up to 7x smaller than uncompressed documents. Smaller documents require less storage space and IO operations, leading to lower storage and IO costs.

In addition to these features, there is more you can do to recognize cost savings on your Amazon DocumentDB workloads. Much like turning off the lights in your home when you’re not there, Amazon DocumentDB allows you to temporarily stop instances in your cluster when they aren’t needed, and start them again when you’re ready to resume. While your cluster is stopped, the cluster storage volume remains unchanged. You are charged only for storage, manual snapshots, and automated backup storage within your specified retention window. You aren’t charged for any instance hours.

As an example, an organization is running a development Amazon DocumentDB environment with a three-node cluster, 24 hours per day. If they stopped the instances on this cluster during nights and weekends, then turned them back on during working business hours, this would result in a reduction of over 73% in total on-demand instance cost per month, as illustrated in the following table.

Table 1: Cost Illustrations

Table 2: Pricing Assumptions

You can enhance this agility through automation with tools like Terraform that give you the ability to deploy architectures that automatically schedule resource provisioning to match the demand of your business and workload.

Automation

Using predefined tags to automate the managing and administering of AWS resources is available in services like Amazon Relational Database Service (Amazon RDS) and Amazon Elastic Compute Cloud (Amazon EC2). For more information, refer to Save costs by automating the start and stop of Amazon RDS instances with AWS Lambda and Amazon EventBridge and Automate starting and stopping AWS instances, respectively.

To help you implement a similar solution for your Amazon DocumentDB resources, we provide an overview of the Terraform project created by the Amazon DocumentDB team that uses the AWS EventBridge Terraform module.

Solution overview

You can download this Terraform project from the amazon-documentdb-samples GitHub repository and run against your Amazon DocumentDB clusters in minutes.

The solution uses two Python scripts run in Lambda: one to stop the instances in a cluster based on the AutoStop tag, and the other to start them based on the AutoStart tag.

Figure 1: Tag Details

This feature uses Amazon EventBridge Scheduler through the templated target for Lambda that calls the Invoke operation by providing each Lambda function’s ARN. The following screenshot shows an example schedule for the AutoStop function.

Figure 2: AutoStop Schedule

The following screenshot shows an example schedule for the AutoStart function.

Figure 3: AutoStart Schedule

This is designed to stop your appropriately tagged Amazon DocumentDB clusters every weekday (Monday–Friday) at 5:30PM UTC and start them again every weekday (Monday–Friday) at 8:00 AM UTC.

All of these services use Amazon CloudWatch to capture logs that you can use for diagnostics or progress monitoring, as shown in the following screenshot.

Figure 4: CloudWatch Logs

Summary

As a fully managed document database, Amazon DocumentDB increases productivity by providing improved performance, scalability, and availability without having to worry about managing the underlying infrastructure. Whether you are migrating an existing application to Amazon DocumentDB or building a new workload, you want to make sure your financial resources are being used properly and you’re getting the most out of the service. Establishing cost-optimization strategies early on maximizes your resource utilization, achieves business outcomes at the lowest possible price point, and meets your functional requirements.

By pairing tools like Terraform with AWS services like EventBridge and Lambda, you can reduce your costs by turning off your Amazon DocumentDB clusters when they are not in use. Refer to the amazon-documentdb-samples GitHub repository to try the solutions discussed in this post.

About the Author

Sonu Kumar Singh is an AWS Solutions Architect, with a specialization in analytics domain. He has been instrumental in catalyzing transformative shifts in organizations by enabling data-driven decision-making thereby fueling innovation and growth. He enjoys it when something he designed or created brings a positive impact. At AWS his intention is to help customers extract value out of AWS’s 200+ cloud services and empower them in their cloud journey.

Cody Allen is a Principal DocumentDB Specialist Solutions Architect based out of Texas. He is passionate about working side by side with customers to solve complex problems, and supporting teammates through mentorship and knowledge transfer. He has spent his career deploying and managing systems, software, and infrastructure for B2B SaaS providers, materiel and logistics suppliers, the U.S. Air Force, and other domestic and international government agencies.