Use the compatibility tool for Amazon DocumentDB (with MongoDB compatibility) to improve migrations

Amazon DocumentDB is a fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads. You can use the same MongoDB 3.6, 4.0, and 5.0 application code, drivers, and tools to run, manage, and scale workloads on Amazon DocumentDB without worrying about managing the underlying infrastructure. As a document database, Amazon DocumentDB makes it simple to store, query, and index JSON data.

Amazon DocumentDB was purpose built from scratch to be cloud native by separating compute and storage. You can take advantage of the quickly distributed underlying storage that grows automatically when needed; an Amazon DocumentDB cluster volume can grow to a maximum size of 128 TiB. Additionally, Amazon DocumentDB automates database clustering and replication, helping to eliminate the need for manual configurations and administration. The overhead of preparing for instance failures, configuring backups, and upgrade plans is handled for you. Amazon DocumentDB automatically promotes a replica instance in the event that your primary instance fails. Backups and restorations are fully managed and you don’t have to create complex upgrade plans. With Amazon DocumentDB, you can focus on developing features and innovating for your users.

In this post, we introduce new feature releases to Amazon DocumentDB and discuss the benefits of the compatibility tool when migrating your workloads to Amazon DocumentDB.

Working backward

At AWS, we work backwards from customer needs to create new products and features; we believe innovation starts there and we invent on the customers’ behalf. For example, we’ve recently added the following updates and features:

• Performance Insights so you can quickly assess the load on your database and determine when and where to take action
• DML auditing to capture read and write events
• Support for $mergeObjects, $reduce, and the Decimal 128 data type
• A JDBC driver to connect from BI tools
• Support for storing, querying, and indexing geospatial data
• Create clones to enable fast creation of a new cluster that uses the same DocumentDB cluster volume and has the same data as the original
• Elastic clusters that provide elastic scale to handle virtually any number of writes and reads, with petabytes of storage capacity
• Client-side Field Level Encryption (FLE) to selectively encrypt sensitive data in-application using AWS Key Management Service (AWS KMS) before it is sent to the database

We have added over 50 additional capabilities to Amazon DocumentDB since its launch, and will continue to work backwards from our customers’ perspective to deliver the capabilities that you need.

Migrate to Amazon DocumentDB

When you consider moving your workload to Amazon DocumentDB, it’s important to consider the supported MongoDB APIs, operations, and data types. Feature compatibility is not a percentage; you must look at the features your application is using and validate compatibility. Auditing your existing workload, either the database or your application code directly, takes time. This is necessary because it is essential to verify that your workload will continue to function and any functional differences are considered. There is no substitution for testing your workload against Amazon DocumentDB to verify a smooth cut-over when performing your migration.

To help expedite and simplify the compatibility checks for your application, the Amazon DocumentDB team has created the Amazon DocumentDB Compatibility Tool, which examines your self-managed MongoDB Server log files or source code from your applications that interact with MongoDB Server to verify that your migration will happen seamlessly and you can modify your application, where identified, to work around unsupported operators.

Compatibility tool

The compatibility tool requires one (or both) of the following:

• Access to your application’s source code
• MongoDB Server profiling logs, capturing as much of the application workload as possible

You can download and run the compatibility tool in minutes.

The compatibility tool and example source code files are available in our GitHub repo.

In this first example, we have a Python script named sample-python-1.py with 100% Amazon DocumentDB compatibility. Running the compatibility tool produces the following output:

$ python3 compat.py --file test/sample-python-1.py

processing file test/sample-python-1.py

Processed 1 files, skipped 0 files

No unsupported operators found

In this second example, our Python script contains an unsupported operator and produces the following output:

$ python3 compat.py --file test/sample-python-2.py

processing file test/sample-python-2.py

Processed 1 files, skipped 0 files

The following 1 unsupported operators were found
$sortByCount | found 1 time(s)

Unsupported operators by filename and line number
$sortByCount | lines = found 1 time(s)
test/sample-python-2.py | lines = [29]

When the compatibility tool finds unsupported operators, it displays the specific files and line numbers of each, making it simple to review the specific code and implement a workaround.

Both of these examples show the compatibility tool analyzing a single file. You can use the --directory parameter to analyze more of the files in a directory and recursively directories within the directory. Other optional command line arguments are shown by running the tool with the -h option.

Note that the compatibility tool is simply one step in a migration journey; it’s important to perform correctness testing, performance testing, and load testing of your application before the actual production migration.

Summary

Amazon DocumentDB interacts with the Apache 2.0 open source MongoDB 3.6, 4.0, and 5.0 APIs. As a result, you can use the same MongoDB drivers, applications, and tools with Amazon DocumentDB with little or no changes. Additionally, you can use the Amazon DocumentDB Compatibility Tool to simplify and expedite your workload migrations to Amazon DocumentDB.

With Amazon DocumentDB, you get a fully-managed document database experience. This means you don’t need to focus on instance failover, database backups and recovery, or software upgrades. You can focus on building your application and innovating for your customers. Visit Get Started with Amazon DocumentDB to begin using Amazon DocumentDB.

About the Authors

Tim Callaghan is a Principal DocumentDB Specialist Solutions Architect at AWS. He enjoys working with customers looking to modernize existing data-driven applications and build new ones. Prior to joining AWS he has been both a producer and consumer of Relational and NoSQL databases for over 30 years.

Cody Allen is a Senior DocumentDB Specialist Solutions Architect based out of Texas. He is passionate about working side by side with customers to solve complex problems, and supporting teammates through mentorship and knowledge transfer. He has spent his career deploying and managing systems, softwares, and infrastructure for B2B SaaS providers, materiel and logistics suppliers, the U.S Air Force, and other government agencies domestic and international.