Building Serverless Event-Driven Applications with MongoDB Realm and Amazon EventBridge

By Igor Alekseev, Partner Solutions Architect, Data & Analytics – AWS
By Andrew Morgan, Staff Engineer – MongoDB

Businesses are faced with ever-increasing complexity of the systems they need to run in order to stay agile and competitive.

The complexity can develop over time as new systems are integrated but old ones are not retired sufficiently fast. It can also develop intentionally, such as when a monolith is decomposed into microservices.

Decoupling individual systems becomes essential in overcoming these challenges. Exchanging events between systems or microservices allows developers to build agile event-driven systems. An added benefit of decoupled systems is the ability to re-architect components to be serverless, which allows organizations to scale easily on demand.

In this post, we will discuss building modern, event-driven serverless applications and how MongoDB Atlas and Realm products complement serverless architectures.

We’ll also review several event-driven serverless reference architectures that leverage MongoDB Atlas, Realm Sync, and Amazon EventBridge—and show you how easy they are to build.

Advantages of Serverless Architectures

The most significant advantage serverless architectures offer is that there is no need to provision or maintain any servers. There are no software or runtime to install, maintain, or administer, and applications can be scaled automatically or by adjusting their capacity.

Typically with serverless architectures, availability and fault tolerance are built in and activated by default. Costs are commensurate with resource consumption, and there is no need to pay for idle capacity or pre/over-provision.

When building serverless applications, you rely on serverless services. Amazon Web Services (AWS) provides a comprehensive portfolio of these services, including AWS Lambda, AWS Fargate, Amazon EventBridge, AWS Step Functions, Amazon Simple Queue Service (SQS), Amazon Simple Notification Service (SNS), Amazon API Gateway, and more. In this post, we will focus on using Amazon EventBridge.

Amazon EventBridge is a serverless event bus that makes it easier to build event-driven applications at scale using events generated from your applications, integrated software-as-a-service (SaaS) applications, and AWS services.

Amazon EventBridge delivers a stream of real-time data from event sources such as Zendesk or Shopify to targets like AWS Lambda and other SaaS applications. You can set up routing rules to determine where to send your data in order to build application architectures that react in real-time to your data sources with event publisher and consumer being completely decoupled.

Why MongoDB for Amazon EventBridge?

MongoDB Atlas is a fully-managed cloud database service. It automates time-consuming administration tasks such as hardware provisioning, software patching, upgrades, and backups. MongoDB Atlas incorporates optimizations and automated best practices.

MongoDB Realm is a fully-managed serverless backend that allows developers to create mobile and web apps. It supports Android, iOS, and web-based apps.

MongoDB Realm can act as both an EventBridge source and target. For example, in the architecture below, user actions become events that propagate via EventBridge to Slack, the final destination.

This mobile-first architecture leverages edge-to-cloud sync, allowing users to work offline while seamlessly syncing their data to MongoDB Atlas when online. See Andrew Morgan’s blog post for detailed step-by-step instructions.

Figure 1 – MongoDB Realm as a source architecture.

EventBridge API Destinations

Now, we’ll discuss how MongoDB Realm can act as the target. Amazon EventBridge recently launched a new feature called API destinations, which allows customers to integrate with any applications that support REST APIs.

API destinations provide a low-code solution by allowing customers to reuse existing APIs while allowing customizations using EventBridge input transformers.

In the architecture below, customers can ingest any event from EventBridge by leveraging Realm webhooks. This is a feature of MongoDB Realm that allows developers to create REST Services. For more details and code samples, refer to this repository.

Figure 2 – MongoDB Realm as a target architecture leveraging EventBridge API destinations.

Using Amazon Kinesis and EventBridge

Another way to ingest data into MongoDB Atlas is via Amazon Kinesis Data Firehose, and you can read more about this ingestion route on the AWS Big Data Blog.

This approach allows you to leverage rich transformations that are available in Amazon Kinesis Data Analytics. For example, you can enrich data upon ingestion by adding a universally unique identifier (UUID). This allows you to track your data immediately, once it arrives in your application.

Another example is implementing a Single View pattern. At the ingestion time, you can aggregate data from multiple sources into a single object to create a single view of anything.

An added benefit of this architecture is that in Amazon Kinesis Data Firehose you can add a splash back to Amazon Simple Storage Service (Amazon S3) for troubleshooting. Once the data is in S3, you can even use it for offline analytics with Atlas Data Lake.

Additionally, Kinesis Data Firehose offers monitoring tools to provide great metrics of your ingestion processes.

Figure 3 – MongoDB Realm as a target architecture leveraging Amazon Kinesis Data Firehose.

Leveraging AWS Lambda to Ingest into MongoDB Atlas

Yet another way to ingest data into MongoDB Atlas is via an EventBridge Lambda route. More details on how to implement this architecture can be found on the MongoDB developer site.

This is a great choice when you need flexibility to transform the data upon ingestion, or need to interact with other AWS services. Your Lambda function can be written in Node.js, Python, Go, Java, and more, but you can also bring your existing code.

AWS Lambda can also run your code to securely access other AWS services via AWS SDK leveraging AWS Identity and Access Management (IAM).

Figure 4 – Using AWS Lambda to ingest data into MongoDB Atlas.

Conclusion

We have described how easy it is to use three different approaches to building serverless event-driven applications with MongoDB Realm, MongoDB Atlas, and Amazon EventBridge.

These components provide proven production-ready building blocks that are cost effective, scalable, and require zero maintenance or idle capacity.

.

.

MongoDB – AWS Partner Spotlight

MongoDB is an AWS Competency Partner. Their modern, general purpose database platform is designed to unleash the power of software and data for developers and the applications they build.

Contact MongoDB | Partner Overview | AWS Marketplace

*Already worked with MongoDB? Rate the Partner

*To review an AWS Partner, you must be a customer that has worked with them directly on a project.