This post is courtesy of Gopalakrishnan Ramaswamy, Solutions Architect

Amazon EventBridge is a serverless event bus that makes it easier to connect applications together using data from your own applications, integrated software as a service (SaaS) applications, and AWS services. It does so by delivering a stream of real-time data from various event sources. You can set up routing rules to send data to targets like AWS Lambda and build loosely coupled application architectures that react in near-real time to data sources.

MongoDB is a document database, which means it stores data in JSON-like documents. It provides a query language and has support for multi-document ACID transactions. MongoDB Atlas is a fully managed MongoDB database service hosted on the cloud. It can be used as a globally distributed database that automates administrative tasks such as database configuration, infrastructure provisioning, patching, scaling, and backups.

With EventBridge, you can use data from MongoDB to trigger workflows for customer support, business operations and more. In this post, I walk through the process of connecting MongoDB Atlas with the AWS Cloud and triggering events from changes in the MongoDB collections data.

Overview

The following diagram shows the high-level architecture of an example scenario to ingest MongoDB data into the AWS Cloud using Amazon EventBridge.

MongoDB stores data records as BSON documents, which are gathered together in collections. A database stores one or more collections of documents.

This walkthrough shows you how to:

1. Create a MongoDB cluster and load sample data.
2. Create a database trigger associated to a collection.
3. Create an event bus in AWS, linked to the partner event source.
4. Create a Lambda function and the associated role with permissions.
5. Create an EventBridge rule and associate it to the Lambda function.
6. Verify the process.

Steps 3–5 create and configure the AWS resources using the AWS Serverless Application Model (AWS SAM). To set up the sample application, visit the GitHub repo and follow the instructions in the README.md file.

Prerequisites

This walkthrough requires:

• An AWS account.
• A MongoDB account.
• The AWS SAM CLI installed and configured on your machine.

Creating a MongoDB Atlas cluster and loading sample data

For detailed steps to create a cluster and load data, see MongoDB Atlas documentation. To create the test cluster:

1. Create a MongoDB Atlas account.
2. Deploy a free tier cluster using these instructions, selecting your preferred cloud provider and Region.
3. Add your trusted connection IP address to the IP access list. This allows to connect to the cluster and access the data.
4. After connecting to your cluster, load sample data into your cluster:
   • Navigate to the clusters view by choosing Clusters in the left navigation pane.
   • Select the cluster, choose the ellipses (…) button, and Load Sample Dataset.

Create MongoDB database trigger

MongoDB database triggers allow you to run server-side logic when a document is added, updated, or removed in a linked cluster. Use database triggers to implement complex data interactions, including updating information in one document when a related document changes or interacting with an external service when a new document is inserted.

1. Sign in to your account and choose Triggers in the left-hand panel.
2. Choose Add Trigger to open the trigger configuration page.
3. Select Database for Trigger Type.
4. Enter a name for the trigger.
5. In the Trigger Source Details section:
   • Select the cluster with sample data loaded (for example, Cluster0) for Cluster Name.
   • For Database Name select sample_analytics.
   • Select customers for Collection Name.
   • Check Insert, Update, Delete, and Replace for Operation Type.
6. In the Function section:
   • For Select An Event Type, Select EventBridge.
   • Enter your AWS Account ID. Learn how to find your account ID in this documentation.
   • Select an AWS Region where the event bus will be created.
7. Choose Save.

Once a MongoDB Atlas trigger is created, it creates a corresponding partner event source in the Amazon EventBridge console. Initially, these event sources show as Pending with no event bus associated to them.

Next, use the AWS SAM template in the GitHub repo to create the event bus, Lambda function, and event rule.

1. Clone the GitHub repo and deploy the AWS SAM template:

   git clone https://github.com/aws-samples/amazon-eventbridge-partnerevent-example
   cd ./amazon-eventbridge-partnerevent-example
   sam deploy --guided

2. Choose a stack name and enter the partner event source name.

The next section explains the steps that are performed by the AWS SAM template.

Creating the event bus

To receive events from SaaS partners, an event bus must be created that is associated to the partner event source:

  PartnerEventBus: 
    Type: AWS::Events::EventBus
    Properties: 
      EventSourceName: !Ref PartnerEventSource
      Name: !Ref PartnerEventSource

The partner event source name and the name of the event bus are derived from the parameter entered when running the template.

Once you create an event bus associated with a partner event source, the status of the partner event source changes to Active. A new event bus with the same name as the partner event source is created. You can see this in the EventBridge console, in Event buses in the left-hand panel.

Creating the Lambda function

The following section of the template creates a Lambda function that is invoked by an event rule:

  myeventfunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: eventLambda/
      Handler: index.handler
      Runtime: nodejs12.x
      FunctionName: myeventfunction

Creating an event bus rule

The following section in the template creates an event rule that triggers the preceding Lambda function. The event pattern used by the rule, selects and routes events to targets.

  myeventrule:
    Type: 'AWS::Events::Rule'
    Properties:
      Description: Test Events Rule
      EventBusName: !Ref PartnerEventSource
      EventPattern: 
        account: [!Ref AWS::AccountId]
      Name: myeventrule
      State: ENABLED
      Targets:
       - 
         Arn: 
           Fn::GetAtt:
             - "myeventfunction"
             - "Arn"
         Id: "idmyeventrule"

Permission is provided to the rule, to invoke Lambda functions. This allows the rule to trigger the associated Lambda function:

  PermissionForEventsToInvokeLambda: 
    Type: AWS::Lambda::Permission
    Properties: 
      FunctionName: 
        Ref: "myeventfunction"
      Action: "lambda:InvokeFunction"
      Principal: "events.amazonaws.com"
      SourceArn: 
        Fn::GetAtt: 
          - "myeventrule"
          - "Arn"         

Verifying the integration

After deploying the AWS SAM template, verify that the EventBridge integration works by inserting test data into the source MongoDB collection. After adding this data, the event is sent to the event bus, which invokes the Lambda function. This is shown in the CloudWatch logs for the event payload.

To verify the deployment:

1. Download and install the MongoDB shell.
2. Connect to MongoDB shell using:

   mongo "mongodb+srv://cluster0.xvo4o.mongodb.net/sample_analytics" --username yourusername

   Replace the cluster name with the cluster you created. Connect to the sample_analytics database, which has the sample data and collections.

3. Next, insert a record into the customers collection with associated the database trigger. In the MongoDB shell, run the following command:

   db.customers.insertOne(
   {
     username:"myuser99",
     name:"Eventbridge Mongo",
     address:"My Address XYZ",
     birthdate:{"$date":"1975-03-02T02:20:31.000Z"},
     email:"mygmailid@gmail.com",
     active:true,
     accounts:[371138,324287,276528,332179,422649,387979],
     tier_and_details:{
        "0df078f33aa74a2e9696e0520c1a828a":{
        tier:"Bronze",
        id:"0df078f33aa74a2e9696e0520c1a828a",
        active:true,
        benefits:["sports tickets"]
       },
      "699456451cc24f028d2aa99d7534c219":{
      tier:"Bronze",
      benefits:["24 hour dedicated line","concierge services"],
      active:true,
      id:"699456451cc24f028d2aa99d7534c219"
     }
     }
     }
   )
   

4. Once the record is successfully inserted:
   • Navigate to CloudWatch in the AWS console and choose Log groups in the left-hand panel.
   • Search for the log group /aws/lambda/myeventfunction and choose the event stream.
   • Expand the log items to reveal the event. This contains the payload that was sent from MongoDB Atlas to EventBridge.

Conclusion

This post demonstrates how to connect MongoDB Atlas data with the AWS Cloud using Amazon EventBridge. EventBridge helps you connect data from a range of SaaS applications using minimal code. It can help reduce operational overhead and build powerful event-driven architectures more easily. For more information about integrating data between SaaS applications, see Amazon EventBridge.

For more serverless learning resources, visit Serverless Land.