AWS Database Blog

Use Amazon QLDB as an immutable key-value store with a REST API and JSON

Amazon Quantum Ledger Database (Amazon QLDB) is a fully managed ledger database service. It’s very helpful when you need to keep your data immutable and maintain a cryptographically verifiable transaction log of all changes made to it. For example, when you need to track data that will be audited in the future, such as financial transactions, invoices data, asset ownership information, and so on.

In this post, we introduce a way to work with Amazon QLDB as an immutable key-value store and use its audit features through a REST API. To illustrate those features, we will introduce a sample use case of tracking invoice data shared by car manufacturers with their dealers.

Use case overview

In our use case, we will see how a car manufacturer can share invoices for new cars they sell to their car dealers. The invoice is digital, and can be easily reprinted and verified by those who have access to the car manufacturer’s REST API that stores data in an Amazon QLDB ledger. The invoices are recorded in Amazon QLDB, but can be shared in various ways or retrieved through the API. The following diagram depicts the process at a high level:

The car manufacturer can share a version of the invoice along with the receipt, and the car dealer can use the receipt to find the version of the invoice in the journal of Amazon QLDB to verify its integrity. In addition, the car dealer can change the invoice status through the REST API to notify the car manufacturer of significant events. For example, car dealer can change the invoice status to “paid” and supply payment transfer details. The car manufacturer will get a notification to check those details against transfer information they receive from their bank. That way, the Amazon QLDB ledger becomes a single source of reference for the most recent invoice data along with all previous changes made to it. Those changes are never deleted and kept as versions of the invoice document. This allows anyone with access rights to retrieve a complete history of changes and compare the details of every change with other data sources such as list of received bank transfers. The feature that automatically keeps all the previous versions of the invoice and have them always available for verification is called “immutability” and is one of the core capabilities of Amazon QLDB.

To deploy the sample application and try out this scenario, follow the steps in the README file in the GitHub repo. Note that the sample code only uses rate limiting to protect the exposed APIs. For production use, you must configure extra protection. For more information, see Protecting your REST API.

In the subsequent sections section of this post, we discuss the details of how the underlying solution works.

Solution overview

The provided REST API exposes functions of the experimental Amazon QLDB key-value store library for Node.js, which helps to use Amazon QLDB as a key-value storage. With this library, we can put and get JSON documents or strings to and from Amazon QLDB and use its standard interface to get document history, retrieve cryptographic receipts, and perform data integrity verification. All interactions with Amazon QLDB are wrapped into library functions for convenience. This is useful to provide an immutable event history when working with frameworks that use a key-value data model. Otherwise, applications should use the Amazon QLDB Driver with the PartiQL query language and Amazon Ion data format. In this solution though, we expose only functions of this sample library through REST API and enable interacting with Amazon QLDB using the JSON document format.

We use a serverless architecture to minimize our potential maintenance and operation spending. The following diagram illustrates the architecture of the solution:

The application architecture contains the following elements:

  1. Amazon API Gateway exposes REST APIs to perform the major operations on the documents stored in Amazon QLDB, and exposes some of the main features of the service, such as document integrity verification and history retrieval.
  2. AWS Lambda hosts function interacting with Amazon QLDB. The function transforms the requests coming from the API Gateway to the function calls of the Amazon QLDB key-value store library for NodeJS and returns the results from Amazon QLDB.
  3. Amazon QLDB is an immutable storage with cryptographic verification that keeps track of all changes to the invoice information and guarantees its integrity.

In addition, we use the AWS Cloud Development Kit (AWS CDK) to set up the infrastructure. The AWS CDK helps us enforce proven defaults when configuring all the necessary cloud resources.

An overview of the key-value store library

We represent invoices as JSON documents that can be inserted into the Amazon QLDB ledger by the value of a single key attribute. Because documents in Amazon QLDB are in Amazon Ion format, the key-value store library automatically converts the invoice in JSON format into Amazon Ion document and back. When you call setValue(key, value) function, in addition to format conversion, the library adds an indexed attribute to store the value of the unique key that we then can use to retrieve the document with getValue(key) function. For example, a JSON invoice document converted into Amazon Ion format may look like this:

{
     _k: "TEST10011",
     date:"2021-05-22",
     billTo:"ABC Car Dealer Pte Ltd",
     paymentStatus:"PENDING",
     carInfo:{
        model:"Honda",
        make:"Jazz",
        year:"2021",
        unitPrice:89000
     },
     quantity:10
}

Note the extra field _k holding an indexed value for a key that we can use to retrieve the document. It is also important to note that the JSON format supports fewer data types than Amazon Ion. The library will not convert the date field of JSON type “String” into an Amazon Ion timestamp type, but will leave it as a string.

Because Amazon QLDB is an immutable ledger database, every time we change a document with the setValue function, instead of overwriting the document the service creates a new revision “on top” of the current one. This allows us to retrieve the history of revisions of the documents we store. We can do so by passing the document key to the getHistory(key, fromDateISOString, toDateISOString) function and optionally specify time range to retrieve revisions we’re interested in.

To retrieve a “receipt” with metadata and cryptographic proofs generated by Amazon QLDB service for the most recent document revision, we can call the getMetadata(key) function by key and then verify that receipt against the Amazon QLDB journal with the verifyMetadata(receipt) function. Besides the most recent version, we can retrieve specific document revisions using the getMetadataByDocIdAndTxId(documentId, transactionId) function which requires the service-generated document ID and transaction ID, which are returned every time we update the document with setValue. With the receipt we can also verify the integrity of the corresponding document revision by first retrieving a complete document revision object with getDocumentRevisionByLedgerMetadata(receipt) and then passing it over to the verifyDocumentRevisionHash(documentRevision) function. That function re-calculates the hash from the document revision data independently from Amazon QLDB and compares it with the hash specified within the metadata part of the complete document revision object to verify that it’s correct.

The key-value store approach is simple to use, but it has some limitations compared to using the Amazon QLDB driver. First, we can only retrieve our JSON or text documents by the value of the key attribute, while the service allows us to use up to five indexed fields per table. Second, we can’t retrieve only a part of the Amazon Ion document stored. We must retrieve the entire document even if our application only needs a small part of it. For more advanced ways of using Amazon QLDB with Node.js see Amazon QLDB driver for Node.js – Quick start tutorial.

More examples of using the key-value store library are available in the GitHub README file.

Conclusion

In this post, we demonstrated a way to expose the functions of experimental Amazon QLDB key-value store library for Node.js as a REST API to store JSON objects or text strings in Amazon QLDB and use its immutability and cryptographic verifiability features.

Happy building on AWS!

About the Authors

Nikolay Vlasov is a Senior Blockchain Architect with AWS Professional Services. Being part of a Global Speciality Practice team, he helps clients worldwide to turn their ideas into Pilots, Minimally Viable Products, and Production-ready systems based on blockchain technology.

Aldred Halim is a Cloud Architect with AWS Professional Services. He works closely with customers in designing architectures and building components to ensure success in their cloud adoption journey.