AWS Database Blog

Introducing Amazon DocumentDB (with MongoDB compatibility) global clusters

Amazon DocumentDB (with MongoDB compatibility) is a fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads. You can use the same MongoDB 3.6, 4.0, or 5.0 application code, drivers, and tools to run, manage, and scale workloads on Amazon DocumentDB without worrying about managing the underlying infrastructure. As a document database, Amazon DocumentDB makes it easy to store, query, and index JSON data.

With the launch of global clusters, Amazon DocumentDB now enables you to deploy a cluster that spans across multiple AWS Regions. Global clusters replicate your data to clusters in up to five Regions with little to no impact on performance. Global clusters provide faster recovery from Region-wide outages and enable low-latency global reads.

A global cluster consists of a primary cluster that allows read and write operations, and up to five secondary clusters in other Regions, which allow read operations. The replication from primary to secondary clusters is unidirectional and allows you to create active-passive deployment. The following diagram illustrates this architecture.

Benefits of Amazon DocumentDB global clusters

Global clusters in Amazon DocumentDB have the following benefits:

  • Disaster recovery from Region-wide outages – While uncommon, global clusters allow you to recover from Region-wide outages in less than 60 seconds. Later in this post, we demonstrate how to promote your secondary cluster to a standalone cluster and recreate a global database in a different Region without data loss.
  • Global reads with low latency – If your application is globally distributed, you can use global clusters to replicate data to other Regions so that your users can read data from the secondary cluster in a Region that is closest to them. Global clusters benefit use cases with a high read to write ratio by serving reads locally with low latency while using the primary cluster for writes.
  • Scalable secondary clusters – The number and type of instances in the primary and secondary clusters don’t need to be the same. You can create secondary clusters with one replica instance and scale up to 16 instances as needed. Scaling instances in Amazon DocumentDB takes less than 10 minutes, regardless of the data volume.
  • High-speed replication across clusters – Global clusters use fast, storage-based physical replication of data from the primary Region to secondary clusters in other Regions. The compute instances provisioned in primary and secondary Regions don’t participate in replication, which frees them up for serving application requests.

Prerequisites

To create an Amazon DocumentDB global cluster, you need an Amazon DocumentDB cluster to serve as primary. You can use an existing Amazon Document cluster or create a new one. When you create a new cluster, the cluster is created as a Regional cluster, by default.

The global cluster must have a primary cluster in one Region and secondary clusters in different Regions.

Create an Amazon DocumentDB global cluster

To set up your global cluster, complete the following steps:

  1. On the Amazon Document DB console, choose Clusters.
  2. Select your Regional cluster.
  3. On the Actions menu, choose Add Region.
  4. For Secondary region, choose a Region.
  5. Enter an identifier for the global cluster and the cluster in the secondary Region.
  6. Choose an instance class and number of instances.
  7. Accept all other defaults and choose Create cluster.

If you disabled encryption for the primary cluster, the secondary clusters are also not encrypted.

The Regional cluster is now upgraded to a global cluster and the data is automatically replicated from the primary cluster to the secondary clusters.

Let’s review the use cases for Amazon DocumentDB global clusters.

Use case 1: Disaster recovery

Prior to global clusters, you could deploy an Amazon DocumentDB cluster with instances that could span three Availability Zones. Although this is highly available, it doesn’t protect against Regional outages. Global clusters allow you to fail over to a secondary cluster in a different Region. This means that your cluster survives the unlikely event of a Regional degradation or a Regional outage.

Prior to the outage, deployment architecture looks like the following diagram.

To fail over an Amazon DocumentDB global cluster, perform the following steps:

  1. Stop application writes to the primary Region.
  2. Identify a secondary cluster that has minimal lag considering your users’ location.
  3. Remove the secondary cluster from the global cluster.

    This action promotes the secondary cluster to a standalone cluster that can accept reads and writes. If you configured additional secondary clusters, we recommend removing them from the global cluster because they will be available for local reads irrespective of primary Region failure.
  4. Reconfigure your application to write to this standalone cluster using its new endpoint.
  5. Add Regions to this standalone cluster to make it a global cluster (as explained earlier in this post).

After failover, the deployment architecture looks like the following diagram.

Use case 2: Low-latency global reads

With Amazon DocumentDB global clusters, you can run your globally distributed applications using a global cluster that spans multiple Regions. Only the primary cluster can perform write operations. Applications that perform write operations connect to the cluster endpoint of the primary cluster. You can find the cluster endpoint for the primary cluster on the Connectivity & security tab of the primary cluster.

You can scale each secondary cluster independently by adding one or more replica instances to serve read-only workloads. You can find the endpoint for the secondary clusters on their respective Connectivity & security tab.

In the following example code, I connect to the primary Region from an Amazon Elastic Compute Cloud (Amazon EC2) instance deployed in the same Region and insert some records. The endpoint is retrieved from the environment variable and must be pre-set using export END_POINT='<endpoint>'.

import pymongo, datetime, os

## Create mongo client with username and password from AWS Secrets Manager
client = pymongo.MongoClient(
    os.environ.get('END_POINT'), 
    27017, 
    username='<userName>', 
    password='<password>', 
    ssl='true', ## TLS Enabled by default
    ssl_ca_certs='rds-combined-ca-bundle.pem',
    replicaSet='rs0', ## Connect as a replica set
    readPreference='secondaryPreferred' ## Reads are sent to replicas
   ## DocumentDB implements the best practice of highly durable writes (write quorum of 4)
   ## w='majority',
   ## j = true
    )
    
db = client.test ##Get the test database
i=1
print("*****************************Write results below*******************************************")
while(i<6):
  currentTime = datetime.datetime.now()
  x = {'_id':i,'time':currentTime}
  db.col.insert_one(x) ## Insert a doc(request routed to Primary)
  print("Inserted document ", i , "at", currentTime ) ## Print to screen
  i = i + 1
  """time.sleep(1/5)"""

client.close()

Next, I read the data inserted to the primary cluster from the cluster in the secondary Region using an EC2 instance in that Region. Set the endpoint as defined before.

import pymongo, datetime, os

## Create mongo client with username and password from AWS Secrets Manager
client = pymongo.MongoClient(
    os.environ.get('END_POINT'), 
    27017, 
    username='<userName>', 
    password='<password>', 
    ssl='true', ## TLS Enabled by default
    ssl_ca_certs='rds-combined-ca-bundle.pem',
    replicaSet='rs0', ## Connect as a replica set
    readPreference='secondaryPreferred' ## Reads are sent to replicas
   ## DocumentDB implements the best practice of highly durable writes (write quorum of 4)
   ## w='majority',
   ## j = true
    )
    
db = client.test ##Get the test database
i=1
print("*****************************Read results below*******************************************")
while (i>0):
  for x in db.col.find():
    print("Retrieved document ", x['_id'], " at", datetime.datetime.now())
    if(x['_id'] == 5) :
      i = -1
client.close()

The following screenshot shows the result of simultaneously running the writer and reader.

The application inserts five records to the primary Region and reads them on the secondary. The data is typically available in the secondary Region in less than a second. With global clusters, you can serve data locally to your users across the globe, as in the following diagram.

The following diagram is a sample deployment architecture for using data locality.

Summary

In this post, I introduced Amazon DocumentDB global clusters. You learned how to fail over in the event of a Regional failover, and how it brings data close to your customer applications in different global Regions.

Do you have follow-up questions or feedback? Leave a comment. I’d love to hear your thoughts and suggestions. To get started, see here.

About the author

Karthik Vijayraghavan is a Senior DocumentDB Specialist Solutions Architect at AWS. He has been helping customers modernize their applications using NoSQL databases. He enjoys solving customer problems and is passionate about providing cost effective solutions that performs at scale. Karthik started his career as a developer building web and REST services with strong focus on integration with relational databases and hence can relate to customers that are in the process of migration to NoSQL.