Introducing transactions in Amazon DocumentDB (with MongoDB compatibility)

With the launch of MongoDB 4.0 compatibility, Amazon DocumentDB (with MongoDB compatibility) now supports performing transactions across multiple documents, statements, collections, and databases. Transactions simplify application development by enabling you to perform atomic, consistent, isolated, and durable (ACID) operations across one or more documents within an Amazon DocumentDB cluster. Common use cases for transactions include financial processes, fulfilling and managing orders, and building multi-player games. In this post, I show you how to use transactions for common uses cases. To get started with Amazon DocumentDB 4.0 and transactions, see Transactions.

ACID transactions

The need for transactions predates the modern computer by centuries. Since people have been keeping books (or tablets, if you will) to track account balances, the need to debit and credit money from one account to another has existed. When money is subtracted from one bank account and added to another bank account, it’s desirable for those two operations to exhibit a set of properties so operations behave as expected without error. Within a database, transactions are often described with the properties of being ACID:

Atomic – All or none. Either all the operations within a transaction complete successfully, or none of them do.
Consistent – A transaction never leaves the database in an inconsistent state.
Isolated – Each transaction operated on the databases doesn’t interfere with other ongoing transactions.
Durable – When the transaction is complete, the changes made to the database are permanent and durable.

Using transactions

To get started with transactions, I discuss four use cases to highlight some of the common scenarios for transactions and show you how to use the API using the mongo shell. To learn more about transactions in Amazon DocumentDB, see Amazon DocumentDB Quotas and Limits.

Use case 1: Multi-statement transaction

One of the canonical use cases for transactions is debiting money from one person’s account and crediting that money in another person’s account. Given that the use case deals with two operations in the database, it’s desirable that the two operations run within a transaction and follow the ACID properties. In this use case, the transaction operates on multiple documents within the same collection to transfer $400 from Sam’s bank account to Joe’s bank account. See the following code:

// To start, drop and create an account collection and insert balances for both Sam and Joe

use test;
db.account.drop();
// true

db.account.insert({"_id": 1, "name": "Sam", "balance": 500.00});
// WriteResult({ "nInserted" : 1 })

db.account.insert({"_id": 2, "name": "Joe", "balance": 10.00});
// WriteResult({ "nInserted" : 1 })

// To start a transaction, create a session and a session object for the account collection 
var mySession = db.getMongo().startSession();
var mySessionObject = mySession.getDatabase('test').getCollection('account');
mySession.startTransaction({readConcern: {level: 'snapshot'}, writeConcern: {w: 'majority'}});

// Within the transaction, debit $400 from Sam’s account
mySessionObject.updateOne({"_id": 1}, {"$inc": {"balance": -400}});
// { "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }

// Similarly, credit Joe’s account with $400
mySessionObject.updateOne({"_id": 2}, {"$inc": {"balance": 400}});
//  { "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }

// Within transaction, you are able to see both of the updates.
mySessionObject.find()
	   // { "_id" : 2, "name" : "Joe", "balance" : 410 }
// { "_id" : 1, "name" : "Sam", "balance" : 100 }

// Outside of the transaction, the updates are not yet visible
db.account.find()
// { "_id" : 1, "name" : "Sam", "balance" : 500 }
// { "_id" : 2, "name" : "Joe", "balance" : 10 }

// Commit the transaction and end the session
mySession.commitTransaction()
mySession.endSession()

// The multi-statement transaction completed successfully the output from both            
// updates is reflected in the database
db.account.find()
   	   // { "_id" : 2, "name" : "Joe", "balance" : 410 }
// { "_id" : 1, "name" : "Sam", "balance" : 100 }

Use case 2: Multi-collection transaction

In addition to running transactions that operate on multiple documents in the same collection, you can run transactions across multiple collections. In the following code, I keep a user’s profile up-to-date with an order count while also placing a new order in the orders collection:

// Drop and create profiles and order collections and insert data
use test;
db.profile.drop();
// true

db.profile.insert({"_id": 1, "name": "Matt", "orders": 22});
// WriteResult({ "nInserted" : 1 })

db.profile.insert({"_id": 2, "name": "Karen", "orders": 5});
// WriteResult({ "nInserted" : 1 })

db.orders.drop();
// true

db.orders.insert({"_id": 1, "orderId": 34333, "product": "shoes", userID: 1});
// WriteResult({ "nInserted" : 1 })

db.orders.insert({"_id": 2, "orderId": 93838, "product": "coffee", userID: 2});
// WriteResult({ "nInserted" : 1 })

// Create a session, session objects for the profile and orders collections, and start a       // transaction
var mySession = db.getMongo().startSession();
var myProfileSessionObject = mySession.getDatabase('test').getCollection('profile');

var myOrdersSessionObject = mySession.getDatabase('test').getCollection('orders');

mySession.startTransaction({readConcern: {level: 'snapshot'}, writeConcern: {w: 'majority'}});

// Insert a new order into the orders collection 
myOrdersSessionObject.insert({"_id": 3, "orderID": 58482, "product": "remote", userID: 2});
// WriteResult({ "nInserted" : 1 })

// Increment the amount of orders in the customer’s profile
myProfileSessionObject.updateOne({"_id": 2}, {"$inc": {"orders": 1}});
	// { "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }

// Commit the transaction and end the session
mySession.commitTransaction()
mySession.endSession()

// Query the end state of the database after the multi-statement transaction completed 
db.profile.find()
// { "_id" : 1, "name" : "Matt", "orders" : 22 }
// { "_id" : 2, "name" : "Karen", "orders" : 6 }

db.orders.find()
	// { "_id" : 1, "orderId" : 34333, "product" : "shoes", "userID" : 1 }
// { "_id" : 2, "orderId" : 93838, "product" : "coffee", "userID" : 2 }
// { "_id" : 3, "orderID" : 58482, "product" : "remote", "userID" : 2 }

Use case 3: Stopping a transaction

The following code shows how to stop an ongoing transaction and how the resulting output doesn’t affect the end state of the database:

// Drop and create a profile collection and insert data
use test;
db.account.drop();
// true

db.account.insert({"_id": 1, "name": "Sam", "balance": 500.00});
// WriteResult({ "nInserted" : 1 })

db.account.insert({"_id": 2, "name": "Joe", "balance": 10.00});
// WriteResult({ "nInserted" : 1 })

// Create a session, session objects for the profile and orders collections, and start a       // transaction
var mySession = db.getMongo().startSession();
var mySessionObject = mySession.getDatabase('test').getCollection('account');
mySession.startTransaction({readConcern: {level: 'snapshot'}, writeConcern: {w: 'majority'}});

// Update the balance for a user
mySessionObject.updateOne({"_id": 2}, {"$inc": {"balance": 50}});
// { "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }

// Query the update to within the transaction
mySessionObject.find()
// { "_id" : 1, "name" : "Sam", "balance" : 500 }
// { "_id" : 2, "name" : "Joe", "balance" : 60 }

// Abort the transaction and end the session
mySession.abortTransaction()
mySession.endSession()

// Confirm that the aborted transaction did not affect the database output
db.account.find()
	// { "_id" : 1, "name" : "Sam", "balance" : 500 }
// { "_id" : 2, "name" : "Joe", "balance" : 10 }

Use case 4: Stopping a transaction due to a write conflict

When attempting to modify a single document from two different transactions, if the data is modified by one transaction, the action can impact other transactions. When one transaction (which we refer to as transaction #2) attempts to modify data that has already been modified by another transaction (transaction #1) but not yet committed, the database throws a WriteConflict error for transaction #2, and transaction #2 is stopped. Because you can’t have multiple transactions each updating the same data, it’s a best practice is to keep transactions small so that they don’t tie up many documents in the database. See the following code:

// Drop and create a profile collection and insert data
use test;
db.account.drop();
// true

db.account.insert({"_id": 1, "name": "Sam", "balance": 500.00});
// WriteResult({ "nInserted" : 1 })

db.account.insert({"_id": 2, "name": "Joe", "balance": 10.00});
// WriteResult({ "nInserted" : 1 })


//To start transaction #1, create a session and a session object for the account collection 
var mySession1 = db.getMongo().startSession();
var mySessionObject1 = mySession1.getDatabase('test').getCollection('account');
mySession1.startTransaction({readConcern: {level: 'snapshot'}, writeConcern: {w: 'majority'}});

//To start transaction #2, create a session and a session object for the account collection 
var mySession2 = db.getMongo().startSession();
var mySessionObject2 = mySession2.getDatabase('test').getCollection('account');
mySession2.startTransaction({readConcern: {level: 'snapshot'}, writeConcern: {w: 'majority'}});

// From transaction #1, update the balance for a user _id:2
mySessionObject1.updateOne({"_id": 2}, {"$inc": {"balance": 50}});
// { "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }

// From transaction #1, query the update to within the transaction
mySessionObject1.find()
// { "_id" : 1, "name" : "Sam", "balance" : 500 }
// { "_id" : 2, "name" : "Joe", "balance" : 60 }

// From transaction #2, attempt to update the account balance for _id:2, which will result in a Write Conflict error
mySessionObject2.updateOne({"_id": 2}, {"$inc": {"balance": -400}});

// 2020-10-04T13:40:59.588+0000 E QUERY    [js] WriteCommandError: Write Conflict  // :
// WriteCommandError({
//	"ok" : 0,
//	"code" : 112,
//	"errmsg" : "Write Conflict",
//	"errorLabels" : [
//		"TransientTransactionError"
//	],
//	"operationTime" : Timestamp(1601818859, 587007)
// })

// End session 2
mySession2.endSession()

// Commit the transaction and end the session
mySession1.commitTransaction()
mySession1.endSession()

// Confirm that the aborted transaction did not affect the database output
db.account.find()
	// { "_id" : 1, "name" : "Sam", "balance" : 500 }
// { "_id" : 2, "name" : "Joe", "balance" : 60 }

Best practices

When developing with transactions, it’s advisable to use transactions for short UPDATE, INSERT, and DELETE use cases as opposed to long-running read queries. In Amazon DocumentDB, similar to MongoDB 4.0, transactions must complete within a minute or the transactions time out. Additionally, the oplog entry for a single transaction must be less than 32 MB. For more information about limits, see Amazon DocumentDB Quotas and Limits. Therefore, if the business logic for a transaction can be split between multiple transactions, it’s advisable to issue multiple smaller transitions than one long transaction.

Lastly, always commit or end a transaction. Transactions that are left open use system resources, cause operations outside of the transaction to block on the completion of the transaction, and can cause write conflicts between transactions, as we showed earlier. For more best practices, see Transactions.

Summary

Support for MongoDB 4.0 compatibility and transactions allows you to perform ACID operations across one or more documents within an Amazon DocumentDB cluster. To learn more, see MongoDB 4.0 Compatibility.

About the author

Joseph Idziorek is a Principal Product Manager at Amazon Web Services.