AWS Compute Blog

Powering Secondary DNS in a VPC using AWS Lambda and Amazon Route 53 Private Hosted Zones

Mark Statham, Senior Cloud Architect

When you implement hybrid connectivity between existing on-premises environments and AWS, there are a number of approaches to provide DNS resolution of both on-premises and VPC resources. In a hybrid scenario, you likely require resolution of on-premises resources, AWS services deployed in VPCs, AWS service endpoints, and your own resources created in your VPCs.

You can leverage Amazon Route 53 private hosted zones to provide private DNS zones for your VPC resources and dynamically register resources, as shown in a previous post, Building a Dynamic DNS for Route 53 using CloudWatch Events and Lambda.

Ultimately, this complex DNS resolution scenario requires that you deploy and manage additional DNS infrastructure, running on EC2 resources, into your VPC to handle DNS requests either from VPCs or on-premises. Whilst this is a familiar approach it adds additional cost and operational complexity, where a solution using AWS managed services can be used instead.

In this post, we explore how you can use leverage Route 53 private hosted zones with AWS Lambda and Amazon CloudWatch Events to mirror on-premises DNS zones which can then be natively resolved from within your VPCs, without the need for additional DNS forwarding resources.

Route 53 private hosted zones

Route 53 offers the convenience of domain name services without having to build a globally distributed highly reliable DNS infrastructure. It allows instances within your VPC to resolve the names of resources that run within your AWS environment. It also lets clients on the Internet resolve names of your public-facing resources. This is accomplished by querying resource record sets that reside within a Route 53 public or private hosted zone.

A private hosted zone is basically a container that holds information about how you want to route traffic for a domain and its subdomains within one or more VPCs and is only resolvable from the VPCs you specify; whereas a public hosted zone is a container that holds information about how you want to route traffic from the Internet.

Route 53 has a programmable API that can be used to automate the creation/removal of records sets which we’re going leverage later in this post.

Using Lambda with VPC support and scheduled events

AWS Lambda is a compute service where you can upload your code and the service runs the code on your behalf using AWS infrastructure. You can create a Lambda function and execute it on a regular schedule. You can specify a fixed rate (for example, execute a Lambda function every hour or 15 minutes), or you can specify a cron expression. This functionality is underpinned by CloudWatch Events.

Lambda runs your function code securely within a VPC by default. However, to enable your Lambda function to access resources inside your private VPC, you must provide additional VPC-specific configuration information that includes VPC subnet IDs and security group IDs. Lambda uses this information to set up elastic network interfaces (ENIs) that enable your function to connect securely to other resources within your private VPC or reach back into your own network via AWS Direct Connect or VPN.

Each ENI is assigned a private IP address from the IP address range within the subnets that you specify, but is not assigned any public IP addresses. You cannot use an Internet gateway attached to your VPC, as that requires the ENI to have public IP addresses. Therefore, if your Lambda function requires Internet access, for example to access AWS APIs, you can use the Amazon VPC NAT gateway. Alternatively, you can leverage a proxy server to handle HTTPS calls, such as those used by the AWS SDK or CLI.

Building an example system

When you combine the power of Route 53 private hosted zones and Lambda, you can create a system that closely mimics the behavior of a stealth DNS to provide resolution of on-premises domains via VPC DNS.

For example, it is possible to schedule a Lambda function that executes every 15 minutes to perform a zone transfer from an on-premises DNS server, using a full zone transfer query (AXFR). The function can check the retrieved zone for differences from a previous version. Changes can then be populated into a Route 53 private hosted zone, which is only resolvable from within your VPCs, effectively mirroring the on-premises master to Route 53.

This then allows your resources deployed in VPC to use just VPC DNS to resolve on-premises, VPC and Internet resources records without the need for any additional forwarding infrastructure to on-premises DNS.

The following example is based on python code running as a Lambda function, invoked using CloudWatch Events with constant text to provide customizable parameters to support the mirroring of multiple zones for both forward and reverse domains.

Prerequisites for the example

Before you get started, make sure you have all the prerequisites in place including installing the AWS CLI and creating a VPC.

  • Region

    Check that the region where your VPC is deployed has the Lambda and CloudWatch Events services available.

  • AWS Command Line Interface (AWS CLI)

    This example makes use of the AWS CLI; however, all actions can be performed via the AWS console as well. Make sure you have the latest version installed, which provides support for creating Lambda functions in a VPC and the required IAM permissions to create resources required. For more information, see Getting Set Up with the AWS Command Line Interface.

  • VPC

    For this example, create or use a VPC configured with at least two private subnets in different Availability Zones and connectivity to the source DNS server. If you are building a new VPC, see Scenario 4: VPC with a Private Subnet Only and Hardware VPN Access.

    Ensure that the VPC has the DNS resolution and DNS hostnames options set to yes, and that you have both connectivity to your source DNS server and the ability to access the AWS APIs. You can create an AWS managed NAT gateway to provide Internet access to AWS APIs or as an alternative leverage a proxy server.

    You may wish to consider creating subnets specifically for your Lambda function, allowing you to restrict the IP address ranges that need access to the source DNS server and configure network access controls accordingly.

    After the subnets are created, take note of them as you’ll need them later to set up the Lambda function: they are in the format subnet-ab12cd34. You also need a security group to assign to the Lambda function; this can be the default security group for the VPC or one you create with limited outbound access to your source DNS: the format is sg-ab12cd34.

  • DNS server

    You need to make sure that you modify DNS zone transfer settings so that your DNS server accepts AXFR queries from the Lambda function. Also, ensure that security groups or firewall policies allow connection via TCP port 53 from the VPC subnet IP ranges created above.

Setting up the example Lambda function

Before you get started, it’s important to understand how the Lambda function works and interacts with the other AWS services and your network resources:

  1. The Lambda function is invoked by CloudWatch Events and configured based on a JSON string passed to the function. This sets a number of parameters, including the DNS domain, source DNS server, and Route 53 zone ID. This allows a single Lambda function to be reused for multiple zones.
  2. A new ENI is created in your VPC subnets and attached to the Lambda function; this allows your function to access your internal network resources based on the security group that you defined.
  3. The Lambda function then transfers the source DNS zone from the IP specified in the JSON parameters. You need to ensure that your DNS server is configured to allow full zone transfers and allow AXFR queries to your DNS server, which happens over TCP port 53.
  4. The Route 53 DNS zone is retrieved via API.
  5. The two zone files are compared; the resulting differences are returned as a set of actions to be performed against Route 53.
  6. Updates to the Route 53 zone are made via API and, finally, the SOA is updated to match the source version.

You’re now ready to set up the example using the following instructions.

Step 1 – Create a Route 53 hosted zone

Before you create the Lambda function, there needs to be a target Route 53 hosted zone to mirror the DNS zone records into. This can either be a public or private zone; however, for the purposes of this example, you will create a private hosted zone that only responds to queries from the VPC you specify.

To create a Route 53 private hosted zone associated with your VPC, provide the region and VPC ID as part of the following command:

aws route53 create-hosted-zone \
--name <domainname> \
--vpc VPCRegion=<region>,VPCId=<vpc-aa11bb22> \
--caller-reference mirror-dns-lambda \
--hosted-zone-config Comment="My DNS Domain"

Save the HostedZone Id returned, since you will need it for future steps.

Step 2 – Create an IAM role for the Lambda function

In this step, you use the AWS CLI to create the Identity and Access Management (IAM) role that the Lambda function assumes when the function is invoked. You need to create an IAM policy with the required permissions and then attach this policy to the role.

Download the mirror-dns-policy.json and mirror-dns-trust.json files from the aws-lambda-mirror-dns-function AWS Labs GitHub repo.

mirror-dns-policy.json

The policy includes EC2 permissions to create and manage ENIs required for the Lambda function to access your VPC, and Route 53 permissions to list and create resource records. The policy also allows the function to create log groups and log events as per standard Lambda functions.

{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Action": [
            "logs:CreateLogGroup",
            "logs:CreateLogStream",
            "logs:PutLogEvents"
        ],
        "Resource": "arn:aws:logs:*:*:*"
    }, {

        "Effect": "Allow",
        "Action": [
            "ec2:CreateNetworkInterface",
            "ec2:DescribeNetworkInterfaces",
            "ec2:DetachNetworkInterface",
            "ec2:DeleteNetworkInterface"
        ],
        "Resource": "*"
    }, {
        "Sid": "Manage Route 53 records",
        "Effect": "Allow",
        "Action": [
            "route53:ChangeResourceRecordSets",
            "route53:ListResourceRecordSets"
        ],

        "Resource": ["*"]
    }]
}

To restrict the Lambda function access, you can control the scope of changes to Route 53 by specifying the hosted zones that are being managed in the format “arn:aws:route53:::hostedzone/Z148QEXAMPLE8V”. This policy can be updated later if additional hosted zone IDs are added.

mirror-dns-trust.json

The mirror-dns-trust.json file contains the trust policy that grants the Lambda service permission to assume the role; this is standard for creating Lambda functions.

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "",
        "Effect": "Allow",
        "Principal": {
            "Service": "lambda.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
    }]
}

Create IAM entities

The next step is to create the following IAM entities for the Lambda function:

  • IAM policy

    Create the IAM policy using the policy document in the mirror-dns-policy.json file, replacing with the local path to the file. The output of the create-policy command includes the Amazon Resource Locator (ARN). Save the ARN, as you need it for future steps.

    aws iam create-policy \
--policy-name mirror-dns-lambda-policy \
--policy-document file://<LOCAL PATH>/mirror-dns-policy.json
  • IAM role

    Create the IAM role using the trust policy in the mirror-dns-trust.json file, replacing with the local path to the file. The output of the create-role command includes the ARN associated with the role that you created. Save this ARN, as you need it when you create the Lambda function in the next section.

    aws iam create-role \
--role-name mirror-dns-lambda-role \
--assume-role-policy-document file://<LOCAL PATH>/mirror-dns-trust.json

    Attach the policy to the role. Use the ARN returned when you created the IAM policy for the
    –policy-arn input parameter.

    aws iam attach-role-policy \
--role-name mirror-dns-lambda-role \
--policy-arn <enter-your-policy-arn-here>

Step 3 – Create the Lambda function

The Lambda function uses modules included in the Python 2.7 Standard Library and the AWS SDK for Python module (boto3), which is preinstalled as part of the Lambda service. Additionally, the function uses the dnspython module, which provides DNS handling functions, and there is also an externalized lookup function.

The additional libraries and functions require that we create a deployment package for this example as follows:

  1. Create a new directory for the Lambda function and download the Python scripts lambdafunction.py and lookuprdtype.py from the aws-lambda-mirror-dns-function AWS Labs GitHub repo. Alternatively, clone the repo locally.
  2. Install the additional dnspython module locally using the pip command. This creates a copy of the require module local to the function. 
    pip install dnspython -t .
  3. Update the lambda_function.py to specify proxy server configuration, if required.
  4. Create a Lambda deployment package using the following command: 
    zip -rq mirror-dns-lambda.zip lambda_function.py \
lookup_rdtype.py dns*

Then, you’ll use the AWS CLI to create the Lambda function and upload the deployment package by executing the following command to create the function. Note that you need to update the commands to use the ARN of the IAM role that you created earlier, as well as the local path to the Lambda deployment file containing the Python code for the Lambda function.

aws lambda create-function --function-name mirror-dns-lambda \
--runtime python2.7 \
--role <enter-your-role-arn-here> \
--handler lambda_function.lambda_handler \
--timeout 60 \
--vpc-config SubnetIds=comma-separated-vpc-subnet-ids,SecurityGroupIds=comma-separated-security-group-ids \
--memory-size 128 \
--description "DNS Mirror Function"
--zip-file fileb://<LOCAL PATH>/mirror-dns-lambda.zip

The output of the command returns the FunctionArn of the newly-created function. Save this ARN, as you need it in the next section.

Configure a test event in order to validate that your Lambda function works; it should be in JSON format similar to the following. All keys are required as well as values for Domain, MasterDns, and ZoneId.

{
    "Domain": "mydomain.com",
    "MasterDns": "10.0.0.1",
    "ZoneId": "AA11BB22CC33DD",
    "IgnoreTTL": "False",
    "ZoneSerial": ""
}

Invoke the Lambda function to test that everything is working; after the function has been invoked, check the file named output to see if the function has worked (you should see a 200 return code). Alternatively, you can test in the AWS console, using the test event to see the log output.

aws lambda invoke \
--function-name mirror-dns-lambda \
--payload fileb://event.json output

Congratulations, you’ve now created a secondary mirrored DNS accessible to your VPC without the need for any servers!

Step 4 – Create the CloudWatch Events rule

After you’ve confirmed that the Lambda function is executing correctly, you can create the CloudWatch Events rule that triggers the Lambda function on a scheduled basis. First, create a new rule with a unique name and schedule expression. You can create rules that self-trigger on schedule in CloudWatch Events, using cron or rate expressions. The following example uses a rate expression to run every 15 minutes.

aws events put-rule \
--name mirror-dns-lambda-rule \
--schedule-expression 'rate(15 minutes)'

The output of the command returns the ARN to the newly-created CloudWatch Events rule. Save the ARN, as you need it to associate the rule with the Lambda function and to set the appropriate Lambda permissions.

Next, add the permissions required for the CloudWatch Events rule to execute the Lambda function. Note that you need to provide a unique value for the –statement-id input parameter. You also need to provide the ARN of the CloudWatch Events rule that you created.

aws lambda add-permission \
--function-name mirror-dns-lambda \
--statement-id Scheduled01 \
--action 'lambda:InvokeFunction' \
--principal events.amazonaws.com \
--source-arn <enter-your-cloudwatch-events-rule-arn-here>

Finally, set the target of the rule to be the Lambda function. Because you are going to pass parameters via a JSON string, the value for –targets also needs to be in JSON format. You need to construct a file containing a unique identifier for the target, the ARN of the Lambda function previously created, and the constant text that contains the function parameters. An example targets.json file would look similar to the following; note that every quote(“) in the Input value must be escaped.

[{
        "Id": "RuleStatementId01",
        "Arn": "<arn-of-lambda-function>",
        "Input": "{\"Domain\": \"mydomain.com\",\"MasterDns\": \"10.0.0.1\",\"ZoneId\": \"AA11BB22CC33DD\",\"IgnoreTTL\": \"False\",\"ZoneSerial\": \"\"}"
}]

Activate the scheduled event by adding the following target:

aws events put-targets \
--rule mirror-dns-lambda-rule \
--targets file://targets.json

Because a single rule can have multiple targets, every domain that you want to mirror can be defined as another target with a different set of parameters; change the ID and Input values in the target JSON file.

Conclusion

Now that you’ve seen how you can combine various AWS services to automate the mirroring of DNS to Amazon Route 53 hosted zones, we hope that you are inspired to create your own solutions using Lambda in a VPC to enable hybrid integration. Lambda allows you to create highly scalable serverless infrastructures that allow you to reduce cost and operational complexity, while providing high availability. Coupled with CloudWatch Events, you can respond to events in real-time, such as when an instance changes its state or when a customer event is pushed to the CloudWatch Events service.

To learn more about Lambda and serverless infrastructures, see the AWS Lambda Developer Guide and the ” Microservices without the Servers” blog post. To learn more about CloudWatch Events, see Using CloudWatch Events in the Amazon CloudWatch Developer Guide.

We’ve open-sourced the code used in this example in the aws-lambda-mirror-dns-function AWS Labs GitHub repo and can’t wait to see your feedback and your ideas about how to improve the solution.