Rotate Active Directory credentials stored in AWS Secrets Manager

In this blog post, I will show you how to use AWS Systems Manager (SSM) Automation to keep a service account’s password synchronized in Microsoft Active Directory (AD) and a Secret in AWS Secrets Manager encrypted with an AWS Key Management Service (KMS) customer managed key (CMK). This blog post uses AWS Secrets Manager, but the concepts can apply to other services, including the AWS Systems Manager Parameter Store.

Introduction

AWS customers use AD as their primary identity provider for users and applications. The challenge with application service accounts stored in AD is they require software, automation, or a person to rotate their passwords. The password needs to be synchronized between AD and the service or application.

The customer scenario that this blog post addresses is rotating the password for the service account used by the recently released Amazon Relational Database Service (Amazon RDS) for SQL Server self-managed AD deployment feature. If you are not familiar with this feature, Amazon RDS for SQL Server uses an AD service account with its credentials stored in an AWS Secrets Manager Secret. It is a recommended practice to rotate service account passwords regularly. At the time of this blog post’s publication, direct integration with AWS Secrets Manager and AD is not available to facilitate this. I will walk you through a solution I came up with to solve this scenario in this blog post.

Solution overview

The solution in this blog post will keep the Amazon RDS for SQL Server’s self-managed AD credentials stored in AWS Secrets Manager in sync with AD. The solution uses a Windows Server 2022 core Amazon Elastic Compute Cloud (Amazon EC2) instance with the AD Administration Tools installed. This Amazon EC2 instance is the target for an AWS SSM State Association scheduled to run every 30 days. The AWS SSM State Association invokes an AWS SSM Run Command document to update an AWS Secrets Manager Secret and AD account to a randomly generated, complex, and long password.

You may ask: Why do I need to use an Amazon EC2 Windows instance? I choose it for simplicity. The only operating system that supports the AD PowerShell cmdlets is Windows. It is true you could use any operating system (OS) that supports Lightweight Directory Access Protocol (LDAP), but the challenge with LDAP writes is that LDAP over SSL is required. You must issue certificates from a trusted Public Key Infrastructure (PKI) solution to the domain controllers to enable LDAP over SSL. While many environments may have this in place, it is not universal. Joining a Windows instance to a domain is the simplest, universally available solution.

The solution AWS CloudFormation template deploys the solution with the following resources. The estimated total cost to run the solution for 1 month is ~$50 (USD) in the AWS US West (Oregon) Region. If you would like to price out all resources deployed by the AWS CloudFormation template, please visit the AWS Pricing Calculator.

• Instance role and profile giving the instance access to the AWS Secrets Manager Secret and AWS KMS Key encrypting the Secret.
• Amazon EC2 Windows instance (t3.medium with 45GiB GP3 Amazon EBS volume) with the AD Administrative tools installed.
• AWS SSM Run Command document to be invoked against the Amazon EC2 Windows instance to update the AWS Secrets Manager Secret and AD credentials.
• AWS SSM Association scheduled to run every 30 days to invoke the AWS SSM Run Command against the Amazon EC2 Windows instance. If you would like to change this to another interval, please update line 1139, the ScheduleExpression parameter in the solution AWS CloudFormation template to a cron expression you desire.

Let’s dive in to the AWS SSM Run Command document (lines 1001-1068 of the solution AWS CloudFormation template). The document contains 4 parameters, RotatedAccountSecretArn, RotatedAccountSecretUserName, RotatedAccountSecretUserNameKey, and RotatedAccountSecretUserPasswordKey. RotatedAccountSecretArn is the ARN of the AWS Secrets Manager Secret you want to rotate. RotatedAccountSecretUserName is the username value for the username key of the AWS Secrets Manager Secret. This parameter allows you to update the username value of the AWS Secrets Manager Secret if desired. RotatedAccountSecretUserNameKey is the username key of the AWS Secrets Manager Secret. RotatedAccountSecretUserPasswordKey is the password key of the AWS Secrets Manager Secret. Since I am using the Amazon RDS for SQL Server self-managed AD deployment feature, the RotatedAccountSecretUserNameKey and RotatedAccountSecretUserPasswordKey will useCUSTOMER_MANAGED_ACTIVE_DIRECTORY_USERNAME and CUSTOMER_MANAGED_ACTIVE_DIRECTORY_PASSWORD respectively.

The action part of the AWS SSM Run Command document (lines 1022-1068 of the solution AWS CloudFormation template) is a PowerShell function named Invoke-PasswordRotation. This function uses 4 parameters: SecretARN, SecretUserName, SecretUserNameKey, and SecretUserPasswordKey, which line up with the 4 parameters of the AWS SSM Run Command document. The PowerShell function first gets the existing values (username and password) of the AWS Secrets Manager Secret. Next, it generates a complex random password of random length between 64 – 128 characters. Then it attempts to update the AWS Secrets Manager Secret values first and then the Active Directory user’s password. If either the AWS Secrets Manager Secret or the Active Directory user’s password cannot update, it will roll back the AWS Secrets Manager Secret to its previous values.

Prerequisites

To follow along in this blog post, you will need the following resources deployed:

• An active self-managed Microsoft AD directory. Open Promote your server to a domain controller if you need to deploy self-managed Microsoft AD. All examples in this blog post use onpremises.local for the self-managed AD domain name.
• An active Amazon EC2 Windows instance joined to the self-managed AD domain (referred to as Onprem MGMT EC2 Instance) with the AD Administration Tools installed.
• An enabled AD service account with a password set.
• An AWS Secrets Manager Secret containing the AD service account credentials.

Figure 1 below shows the prerequisites with the solution components used in this blog post.

Figure 1: Prerequisites & Solution

Steps to deploy the prerequisites AWS CloudFormation template

I am providing you with an AWS CloudFormation template to deploy the prerequisites in a new Amazon Virtual Private Cloud (Amazon VPC). Download the template to your computer.

Here are the steps to deploy the AWS CloudFormation template. It should take around 60 minutes for the CloudFormation template to deploy. The estimated total cost to run the resources for 24 hours is ~$60 (USD) in the AWS US West (Oregon) Region. If you would like to price out all resources deployed by the AWS CloudFormation template, please visit the AWS Pricing Calculator.

1. Sign in to the AWS CloudFormation console.

2. Select Create stack.

3. In the Specify template section, select Upload a template file, select Choose file, browse to, and select the AWS CloudFormation Template you downloaded from here, and select Next.

4. In the Specify stack details page, set the following and select Next.

a. Stack name: Service-Account-Rotator-Prerequisites.

b. Parameters: Leave the default settings.

5. In the Configure stack options page, leave the default settings, and select Next.

6. In the Review stack page, check I acknowledge that AWS CloudFormation might create IAM resources with custom names and select Submit.

If you plan on using your own environment, make sure you use the information for your environment.

Step-by-step instructions

I wrote the following instructions with the expectation the prerequisites are in place. The instructions are based on the resources deployed by the AWS CloudFormation template I provided. Download the solution AWS CloudFormation template to your computer.

Step 2: Deploy the solution via AWS CloudFormation

To deploy the solution AWS CloudFormation template, I have provided the steps below. It should take around 20 minutes for the CloudFormation template to deploy. If you would like to price out the resources deployed by the AWS CloudFormation template, please visit the AWS Pricing Calculator.

1. Sign in to the AWS CloudFormation console.

2. Select Create stack.

3. In the Specify template section, select Upload a template file, select Choose file, browse to, and select the solution AWS CloudFormation template you downloaded from here, and select Next.

4. In the Specify stack details page, set the following and select Next.

a. Stack name: Service-Account-Rotator.

b. Parameters:

i. SSM Parameter for Latest AMI ID: Leave the default value of /aws/service/ami-windows-latest/TPM-Windows_Server-2022-English-Full-Base.

ii. EC2 Instance Type: Leave the default value of t3.medium.

iii. EBS KMS Encryption Key: Leave the default value of alias/aws/ebs.

iv. Task Instance Subnet ID: ID of subnet where the solution instance was deployed to. This is the value of the Subnet1Id key gathered in Step 1: Gather information.

v. Task Instance VPC ID: ID of the VPC where the solution instance was deployed. This is the value of the VPC key gathered in Step 1: Gather information.

vi. Task Instance NetBIOS Name: RDS-Svc-Act-Rtr.

vii. AD Domain DNS Name: onpremises.local.

viii. AD Domain NetBIOS Name: ONPREMISES.

ix. Task Instance DNS Resolver IP 1: DNS resolver IP address for onpremises.local. This is the value of the DomainControllerInstancePrivateIP key gathered in Step 1: Gather information.

x. Task Instance DNS Resolver IP 2: DNS resolver IP address for onpremises.local. Since the prerequisite template only deployed a single instance, use 10.0.0.2.

xi. Secret ARN for Configuration Credentials: ARN of Secret that contains credentials to domain join and configure the solution instance. This is the value of the SetupAccountSecretArn key gathered in Step 1: Gather information.

xii. KMS Key ARN for Secret used for Configuration Credentials: ARN of the KMS key used to encrypt the Secret that contains credentials used to domain join and configure the solution instance. This is the value of the aws/secretsmanager AWS KMS key gathered in Step 1: Gather information.

xiii. Secret ARN for Credentials to be Rotated: ARN of Secret that contains credentials to be rotated. This is the value of the RotatedAccountSecretArn key gathered in Step 1: Gather information.

xiv. KMS Key ARN for Secret used for Rotated Credentials: ARN of the KMS key used to encrypt the Secret that contains credentials to be rotated. This is the value of the RotatedAccountSecretKeyArn key gathered in Step 1: Gather information.

xv. Credentials to be Rotated Username: RdsServiceAccount.

xvi. Credentials to be Rotated Username Key: CUSTOMER_MANAGED_ACTIVE_DIRECTORY_USERNAME.

xvii. Credentials to be Rotated Password Key: CUSTOMER_MANAGED_ACTIVE_DIRECTORY_PASSWORD.

5. In the Configure stack options page, leave the default settings, and select Next.

6. In the Review stack page, check I acknowledge that AWS CloudFormation might create IAM resources with custom names and select Submit.

After about 20 minutes, it should successfully deploy your template. If the template cannot deploy, you will want to review the following locations for the failure reason:

• The AWS CloudFormation console stack details.
• The AWS Systems Managed Automation execution details.
• The logs in AWS CloudWatch Logs console. The SSM Automation deployment steps write all standard output to a log group with a named/aws/Rotator/<StackName>.

Step 3: Validate the solution is rotating the password in AD and in the Secret

Now that you have deployed the solution, let’s validate the solution is rotating the Amazon RDS for SQL Server service account credentials as expected. If for any reason the solution cannot rotate the AD password or the AWS Secrets Manager Secret password value, it will revert them to the previous value.

1. Open the AWS Systems Manager Fleet Manager – Remote Desktop console. Select Add new session, select the Onprem MGMT EC2 Instance node, and select Add.

2. Select User credentials and enter the following parameters and select Connect.

a. Username: onpremises\Administrator.

b. Password: In the OnPremAdministratorSecret Secret (from Step 1: Gather information).

3. Right-click on Start and select Windows PowerShell (Admin).

4. In the elevated Windows PowerShell window, run the PowerShell command shown below .

Get-ADUser -Identity ‘RdsServiceAccount’ -Properties ‘PasswordLastSet’ | Select-Object -ExpandProperty ‘PasswordLastSet’

The script output should look like:

Thursday, June 29, 2023 3:16:33 PM

5. Open the AWS Systems Manager Run Command console.

6. Select Command History, look for and select the Command ID with the document name of Rotator-Document-<StackName>.

7. Note of the Finish time. It should be very close or match the time from the PowerShell command above. Figure 2 below gives an example of what to look for.

Figure 2: Run Command Status

8. In the same window, select Instance ID, and expand the Output section. This is where you can see the output of the AWS SSM Run Command that rotated the AWS Secrets Manager Secret credential. Figure 3 below gives an example of what successful output will look like.

Figure 3: Run Command Output

Cleanup

If you decide you would like to remove the resources deployed in your account from this blog post, perform the following steps:

1. Delete both AWS CloudFormation stacks that deployed the prerequisites and rotator solution for this blog post. You can find here the instructions for deleting stacks.

Summary

In this blog post, I showed you how to set up an automated solution to rotate and synchronize AD credentials stored in an AWS Secrets Manager Secret. This solution works with self-managed AD and AWS Managed Microsoft AD. You could apply this solution for the AD service account used to seamlessly join Amazon EC2 Linux instances to an AWS Managed Microsoft AD directory. The Amazon EC2 instance deployed with this solution could have multiple AWS SSM State Manager associations configured to rotate multiple credentials.

For more information on the Amazon RDS for SQL Server self-managed AD feature, please refer to the Amazon RDS for SQL Server Administrator Guide.

AWS can help you assess how your company can get the most out of cloud. Join the millions of AWS customers that trust us to migrate and modernize their most important applications in the cloud. To learn more on modernizing Windows Server or SQL Server, visit Windows on AWS. Contact us to start your migration journey today.