Evaluate Amazon DocumentDB (with MongoDB compatibility) configurations using AWS Config

It’s common practice for organizations to define compliance and standards for all applications and software they interact with, such as databases, storage, network, and compute. The key drivers are to comply with different regulations to achieve security and audit certifications according to the domain they operate in. AWS Config allows you to create rules and configure resources so they comply with the standards enforced by your organization. These rules are configured to be evaluated either periodically or every time a configuration change occurs to validate if the change has violated the rule.

AWS Config enables you to assess, audit, and evaluate the configurations of your AWS resources. AWS Config continuously monitors and records your AWS resource configurations and allows you to automate the evaluation of recorded configurations against desired configurations. With AWS Config, you can review changes in configurations and relationships between AWS resources, dive into detailed resource configuration histories, and determine your overall compliance against the configurations specified in your internal guidelines. This enables you to simplify compliance auditing, security analysis, change management, and operational troubleshooting.

Amazon DocumentDB (with MongoDB compatibility) is a fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads. You can use the same MongoDB 3.6, 4.0 or 5.0application code, drivers, and tools to run, manage, and scale workloads on Amazon DocumentDB without worrying about managing the underlying infrastructure. As a document database, Amazon DocumentDB makes it easy to store, query, and index JSON data.

To improve the ability to continuously monitor and notify when configurations in Amazon DocumentDB deviate from best practices, you can create AWS Config rules and trigger notifications immediately when the cluster modification violates compliance and standards predefined by your organization. In this post, we discuss using AWS Config to evaluate Amazon DocumentDB configurations by using out-of-the-box managed rules and defining custom rules. We also discuss taking proactive measures to automatically correct any deviations and trigger notifications within minutes when Amazon DocumentDB configurations are altered.

Solution overview

The following architecture explains how to set up AWS Config to evaluate Amazon DocumentDB configurations. AWS Config checks for compliance rules such as backup retention policy changes and triggers a notification by sending an event to Amazon EventBridge, a serverless event bus. This event is created when a noncompliant configuration change is detected, such as a new cluster created with a backup retention period that doesn’t match an organization’s Recovery Point Objective (RPO) requirements. Because we have a handle on the event, we can proactively remediate the noncompliant action using an AWS Lambda function and trigger a notification informing the stakeholders about the change. In the backup retention policy example, our Lambda function modifies the cluster and updates its backup retention policy to be compliant again.

AWS Config also allows you to configure manual or automatic remediation by publishing the noncompliant event to an Amazon Simple Notification Service (Amazon SNS) topic. In this post, we use EventBridge due to the wider range of targets we could send the event to if required.

Prerequisites

If you’re using AWS Config for the first time in an AWS Region, you must set it up. For instructions, see Setting Up AWS Config with the Console. You can skip the rule selection because we do that in the following sections.

Amazon DocumentDB uses operational technology that is shared with Amazon Relational Database Service (Amazon RDS). As a result, we define Amazon DocumentDB clusters and instances as Amazon RDS resources when using AWS Config.

All the resources that are required for this solution are included as part of an AWS Cloud Development Kit (AWS CDK) application that automatically provisions them and whose source code can be found in the following GitHub repository. Although in the following sections we explain how to manually create and configure the resources using the AWS Management Console, we recommend doing it through the code sample provided.

Get started with AWS CDK

To help get started with AWS Config and Amazon DocumentDB, we created a repository that uses an AWS CDK framework with Typescript. It contains all the AWS Config rules we mention throughout this post so Amazon DocumentDB resources can be evaluated to meet regulatory and compliance requirements. It also has examples for setting up automated notification and remediation of noncompliant config changes.

To deploy this AWS CDK application to your account, follow these steps:

Enable AWS Config for the account and Region where rules are deployed to.
Only for the first time, bootstrap the target environment (AWS account and Region) where you’re provisioning the AWS CDK app resources:

export ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)
export AWS_REGION=$(aws configure get region)

cdk bootstrap aws://$ACCOUNT_ID/$AWS_REGION

From the root of the project, install dependencies, compile, and deploy:

npm install
npm run build
cdk deploy

After you confirm the security-related changes, the creation of the AWS CloudFormation stack starts. You can monitor its progress directly from the terminal or on the AWS CloudFormation console.

You can easily modify (and extend) this repository to apply your own requirements and deploy these rules and automated responses to your accounts.

Resource configuration and history

As previously mentioned, AWS Config audits every configuration change that occurs for the resources it supports, including Amazon DocumentDB clusters and instances. These changes are available through the AWS Config console (and APIs) through the compliance history timeline, and can be used to understand how a resource has evolved over time and which actions caused the resource to become noncompliant against the rules defined by your organization.

Let’s explore how we can review the configuration history of an Amazon DocumentDB cluster:

On the AWS Config console, choose Resources in the navigation pane.
Choose the cluster you want to review. To find it more easily, you can filter the list of resources by type: AWS RDS DBCluster.

In the resource details, you can find a point-in-time view of the cluster through what is called its configuration item, which is expressed by a JSON object that includes resource metadata such as attributes or relationships with other resources. You can also find what rules are applicable for this cluster alongside its compliance status.

Choose Resource Timeline.

A timeline of events that affected the cluster’s configuration is displayed. These events belong to either AWS CloudTrail, configuration, and compliance events.

In the following screenshot, CloudTrail captured an action that modified the cluster, the corresponding configuration change, and as a consequence, a reevaluation of the available rules.

You can expand any event to find out more information about it.

Define rules for compliance and standards

In this section, we discuss defining our managed and custom AWS Config rules.

Managed rules

In this post, we use AWS Config managed rules for evaluating deletion protection and storage encryption configuration of an Amazon DocumentDB cluster. These options are enabled by default and it’s recommended to not disable these for production clusters. When we define AWS Config rules, we can enable proactive remediation or notification if a cluster is created with encryption or delete protection disabled. To create a managed rule, complete the following steps:

On the AWS Config console, choose Rules.
Choose Add rule.
Select Add AWS managed rule.
Search for storage-encrypted.
Select rds-storage-encrypted.
Choose Next.
Enter a name and description for the managed rule.
For Scope of changes, select Resources.
Choose Next.
In the Review and create section, choose Add rule to create the managed rule.

If your Region currently has an Amazon DocumentDB cluster that has storage encryption enabled, you see the compliance status as evaluated by the rule.

Create an Amazon DocumentDB cluster with encryption at rest disabled and review the preceding AWS Config rule.

This rule is evaluated at the instance level and when noncompliant, it refers to as many instances as you have in the cluster.

Choose the rule link to see the resources that aren’t compliant.
Repeat the preceding steps to create a new managed rule for deletion protection.

Custom rules

We use custom rules for evaluating those settings that aren’t available with managed rules, such as cluster parameter group, backup retention period, and instance type configurations. In Amazon DocumentDB, you can use a cluster parameter group to enable or disable audit logs, profiler logs, TLS, and ttl_monitor. AWS Config custom rules allow you to define rules to validate if the cluster parameter group adheres to the predefined values.

Custom rules logic is provided through Lambda functions. Therefore, we need to create the function that evaluates whether our DocumentDB cluster complies or not with the parameter group defined by our organization.

Lambda allows you to create functions from a blueprint so you have a starting point for common use cases. One of them is config-rule-change-triggered, which provides a code sample to implement AWS Config custom rules that evaluate changes as they happen.

Follow these steps to create the function:

On the Lambda console, choose Functions.
Choose Create function.
Select Use a blueprint and filter by config-rule.

Two results are displayed: one for rules that are triggered periodically (config-rule-periodic) and one for rules that are triggered when configuration changes happen (config-rule-change-triggered). Both blueprints are written in JavaScript.

Choose the blueprint config-rule-change-triggered and choose Configure.
Enter a name for the function and a name for the associated AWS Identity and Access Management (IAM) role.
Leave Create the execution role from AWS policy templates selected because this gives the function the required permissions to interact with AWS Config.
Choose Create function.

After the Lambda function is created, it’s time to modify the default code sample and provide the logic of our rule.

On the Code tab, choose (double-click) index.js to open its content.

Most of the JavaScript functions in the file deal with generic tasks that are common to the majority of custom rules such as validations or passing compliance results back to AWS Config. For most scenarios, the only function that we must modify to run our compliance check is evaluateChangeNotificationCompliance.

To make sure that we’re using the right cluster parameter group for our Amazon DocumentDB cluster, replace its body with the following code:

// Evaluates whether the cluster parameter group is the one provided to the rule as a parameter
async function evaluateChangeNotificationCompliance(configurationItem, ruleParameters) {
  checkDefined(configurationItem, 'configurationItem');
  checkDefined(configurationItem.configuration, 'configurationItem.configuration');
  checkDefined(configurationItem.configuration.dbclusterParameterGroup, 'configurationItem.configuration.dbclusterParameterGroup');
  checkDefined(ruleParameters, 'ruleParameters');

  if (ruleParameters.desiredClusterParameterGroup === configurationItem.configuration.dbclusterParameterGroup) {
    return 'COMPLIANT';
  }
  return 'NON_COMPLIANT';
}

In the preceding snippet, after we validate we have all the required information to run our compliance check, we simply compare the new cluster parameter group with the one set by our organization and that is made available to our code through the desiredClusterParameterGroup property. The desired cluster parameter group gets set when creating the corresponding custom rule, which we do next.

Choose Deploy to save the changes.

Now that the Lambda function is created, we can proceed with the associated custom rule.

On the AWS Config console, choose Rules.
Choose Add rule.
Select Create custom rule.
Choose Next.
Enter a name and description for the custom rule.
For AWS Lambda function ARN, enter the ARN of the function that has the rule definition.

To get the ARN, navigate to the Lambda console, go to the function’s details, and choose Copy ARN. If you’re using the code from the GitHub repository mentioned earlier, the function name for the following example contains ClusterParameterGroup.

For Trigger type, select When configuration changes so that the rule only runs when the cluster attributes are modified.
For Scope of changes, select Resources.
For Resource category, choose AWS RDS DBInstance.
For Parameters, create a new parameter called desiredClusterParameterGroup (the Lambda function relies on it as we defined earlier) and for the value, enter the name of the cluster parameter group you want your clusters to have so they’re compliant. You can create a cluster parameter group and assign it to the DocumentDB cluster.
Choose Next.

You can be more specific when specifying which resources this rule applies to by using tags as the scope of changes instead of resources.

Choose Add rule to create the custom rule.
Verify the compliance rule by changing the cluster parameter group value (for example, disable TLS).
Choose the rule link to see the resources that aren’t compliant.

The resources in scope are filtered by Noncompliant by default. You can also choose a resource to see the status for various AWS Config rules defined for it.

You can repeat these steps to create as many custom rules as your organization needs to comply with. You can use the examples to check backup retention period and instance type configuration in the code repository.

Configuration changes

Every time a compliance check to your rules fails, AWS Config publishes an event to the default bus on EventBridge. Interested consumers can subscribe to these events and act accordingly.

Subscriptions are configured in EventBridge through rules that attempt to match the event against a given pattern. When successful, they relay the event to one or many targets or consumers.

Next, we discuss how we can add notifications and remediation to complement both managed and custom rules created previously in this post.

Notifications

Whenever a rule becomes noncompliant, we can notify interested stakeholders by publishing a message to an SNS topic.

First, we create the SNS topic:

On the Amazon SNS console, choose Topics.
Choose Create topic.
Choose Standard as the type.
Enter a name and leave other fields as default.
Choose Create topic.

From the topic’s detail page, you can subscribe endpoints that are notified when a message is published to the topic. Amazon SNS supports different protocols for these endpoints.

For this post, we use email as the target.

You can choose other targets based on your own requirements (for instance, you may want to notify a channel in Slack, which you can do through a Lambda function).

We’re now ready to configure an EventBridge rule.

On the EventBridge console, choose Rules.
Choose Create rule.
Enter a name and description to the rule.
Choose Event Pattern.
Choose Custom pattern.
Enter the following JSON expression:

{
  "detail-type": ["Config Rules Compliance Change"],
  "source": ["aws.config"],
  "detail": {
    "messageType": ["ComplianceChangeNotification"],
    "newEvaluationResult": {
      "evaluationResultIdentifier": {
        "evaluationResultQualifier": {
          "configRuleName": [{
            "prefix": "documentdb-"
          }]
        }
      },
      "complianceType": ["NON_COMPLIANT"]
    },
    "resourceType": ["AWS::RDS::DBCluster", "AWS::RDS::DBInstance"]
  }
}

The preceding pattern matches those events generated by AWS Config (source property) where there was a change in the compliance status for a resource (detail-type). We can narrow down further to the events we’re interested in by specifying resource type, noncompliant status, and rules with a name that starts with documentdb-. Using a fixed prefix for Amazon DocumentDB rules allows us to discard other possible rules that are triggered for Amazon RDS resources.

Leave the event bus as the AWS default event bus because this is the bus where AWS services and resources publish events to.
In the Select targets section, choose SNS topic on the drop-down menu and the topic we created earlier.

Behind the scenes, EventBridge is granted permission to publish messages to the topic through the SNS topic access policy.

Leave the remaining settings as default and choose Create.

Remediation

When the cluster parameter group rule becomes noncompliant, we can trigger a Lambda function that automatically remediates the issue by setting back the parameter group to the expected value.

We have to create a function that is responsible for resetting the desired parameter group, as shown in the following code:

const aws = require('aws-sdk');
const docDb = new aws.DocDB();

exports.handler = async event => {
  try {
    const desiredClusterParameterGroup = process.env.DESIRED_CLUSTER_PARAMETER_GROUP;

    if (!desiredClusterParameterGroup) {
      throw new Error('Desired cluster parameter group not found');
    }

    const {detail: {resourceId}} = event;
    const dbClusterIdentifier = await getDbClusterIdentifier(resourceId);
    const params = {
      DBClusterIdentifier: dbClusterIdentifier,
      DBClusterParameterGroupName: desiredClusterParameterGroup
    };

    await docDb.modifyDBCluster(params).promise();
    return;

  } catch (e) {
    console.log('There has been an error', e);
    throw e;
  }
};

async function getDbClusterIdentifier(resourceId) {
  try {
    const {DBClusters: clusters} = await docDb.describeDBClusters().promise();  
    const {DBClusterIdentifier: dbClusterIdentifier} = clusters.find(c => c.DbClusterResourceId === resourceId);

    if (!dbClusterIdentifier) {
      throw new Error(`Cluster with resourceId=${resourceId} not found`);
    }

    return dbClusterIdentifier;

  } catch (e) {
    console.log(e);
    throw e;
  }
}

This function simply updates the noncompliant cluster to use the parameter group that matches our organization needs and is provided through an environment variable. It also finds the cluster identifier that the modifyDBCluster expects because the event generated by AWS Config (the input of our handler function) only provides its resource ID.

Based on the actions performed by the function, its execution IAM role must have a policy attached that grants permission over rds:DescribeDBClusters and rds:ModifyDBCluster operations.

After the function is created, we can set up a rule and target in EventBridge.

Complete the steps in the preceding section but enter the following event pattern, so we only trigger the function when the parameter group rule is noncompliant:

{
  "detail-type": ["Config Rules Compliance Change"],
  "source": ["aws.config"],
  "detail": {
    "messageType": ["ComplianceChangeNotification"],
    "newEvaluationResult": {
      "evaluationResultIdentifier": {
        "evaluationResultQualifier": {
          "configRuleName": ["documentdb-cluster-parameter-group"]
        }
      },
      "complianceType": ["NON_COMPLIANT"]
    },
    "resourceType": ["AWS::RDS::DBCluster"]
  }
}

Select Lambda function as a target and choose the name of your function.

As with the SNS topic, EventBridge is automatically granted permission to invoke the function through its resource policy.

Choose Create.

It’s now time to test our automated remediation. Go to one of your Amazon DocumentDB clusters and modify the parameter group so it doesn’t match the one specified by the rule (you have to reboot the instances to make the change effective). It first shows a noncompliant status and becomes compliant again after a few moments. You can visit the resource timeline to better understand what happened and how the remediation took place without manual intervention.

When applying remediation, be aware of which rules can be fixed by modifying the cluster and which ones require a new cluster to be created. For example, encryption at rest can’t be enabled for an existing cluster, therefore no remediation can be automatically performed (although you can prevent this from happening using IAM policies); we can only notify about the compliance failure. However, if deletion protection is disabled, we can enable it again following a similar approach to what we showed earlier for the parameter group use case.

Summary

In this post, we discussed using AWS Config to evaluate Amazon DocumentDB configurations against best practices. We also discussed how to use managed and custom rules in AWS Config along with EventBridge and Lambda functions to proactively remediate any noncompliant events.

You can automatically provision all the resources and configurations mentioned throughput this post to an AWS account and Region using an AWS CDK application that can be found in the following repository under the aws-samples organization in GitHub.

About the Authors

Jesus Bernal is a Solutions Architect at Amazon Web Services. Jesús is based in Madrid, Spain, where he works helping startups achieve their business goals while building secure, highly available and scalable solutions together with keeping go-to-market times to a minimum. Outside work, you can find him watching football, training for the next half-marathon or reading the latest best seller.

Karthik Vijayraghavan is a Senior DocumentDB Specialist Solutions Architect at AWS. He has been helping customers modernize their applications using NoSQL databases. He enjoys solving customer problems and is passionate about providing cost effective solutions that performs at scale. Karthik started his career as a developer building web and REST services with strong focus on integration with relational databases and hence can relate to customers that are in the process of migration to NoSQL.

AWS Database Blog