Automate SCIM provisioning from Active Directory to IAM Identity Center

In this blog post, we will show you how to build and deploy a custom solution to automate the process of provisioning users and groups from Microsoft Active Directory (AD) to AWS IAM Identity Center using the System for Cross-domain Identity Management (SCIM) protocol.

Introduction

Many organizations manage their user identities using AD and rely on external identity providers to provide federated access to Amazon Web Services (AWS) via integration with IAM Identity Center. Some identity providers such as Active Directory Federation Service (AD FS), Shibboleth, and F5 either do not support SCIM or have an incompatible SCIM implementation. The end result is that you must manually provision users and groups from AD into IAM Identity Center. We will walk you through the steps to automate this manual provisioning process when using such identity providers.

Solution overview

This solution is based on a serverless architecture using an AWS Lambda function that runs a Python script to automate the synchronization of users and groups from AD to IAM Identity Center. The following are the key steps involved in the synchronization.

1. The Lambda function establishes a connection to your AD server and retrieves the users and groups.
2. The script then processes the data, mapping the AD attributes to the corresponding IAM Identity Center attributes.
3. The script synchronizes the users and groups to IAM Identity Center, performing create, update, and delete operations as needed.

The solution leverages asynchronous coding to enable an efficient synchronization process, especially when dealing with large directories. It also provides support for Lightweight Directory Access Protocol over SSL (LDAPS) to securely connect to your AD server.

The architecture (Figure 1) utilizes AWS Secrets Manager and AWS Key Management Service (AWS KMS) to store and encrypt sensitive information, such as the AD service account credentials. You can invoke the Lambda function manually or schedule it to run periodically to maintain consistency between AD and IAM Identity Center, which is crucial for effective identity and access management within your cloud environment.

The key benefits of this solution include:

• Reduced manual effort in provisioning and managing users and groups.
• Improved consistency between AD and IAM Identity Center, ensuring accurate access controls.
• Support for organizations using both AD and IAM Identity Center as their identity management systems.

Figure 1 – Solution architecture

Prerequisites

The following prerequisites are required for this walkthrough.

• An organization set up in AWS Organizations.
• The management account of the organization where you have enabled an organization instance of IAM Identity Center. The instance needs to be configured with an external identity provider as your identity source, with automatic provisioning enabled and the SCIM endpoint and access token available.
• A member account in the organization with an Amazon Virtual Private Cloud (Amazon VPC) that has two private subnets to access the AD domain controller(s), the SCIM endpoint, the Secrets Manager endpoint, and the Identity Store endpoint. (It is possible to use the management account itself that has the aforementioned resources, but it is not recommended.)
• Credentials (username and password) for a read-only service account in AD that can access users and groups to provision.

Walkthrough

The following steps provide a walkthrough of the solution.

1. Create resources in the chosen account – Deploy an AWS CloudFormation stack using the ad-sync.yaml template. This will create the necessary resources, including the Lambda function and the supporting AWS services.
2. Create and add a Lambda layer – Upload the required dependencies (e.g., requirements.txt, CA certificates file) as a zip file to an Amazon S3 bucket and then create a Lambda layer using the uploaded zip file.
3. Invoke the Lambda function – Manually test the Lambda function to perform the initial synchronization of users and groups from AD to IAM Identity Center. Review the logs in Amazon CloudWatch Logs to ensure the synchronization was successful.
4. [Optional] Set up custom SCIM token expiration notification – Deploy a CloudFormation stack in the management account using the ad-sync-scim-cross-account-event-notification.yaml template. This will set up the necessary resources to receive notifications about SCIM token expirations.

Create resources in the member account

1. Log into the chosen account and navigate to CloudFormation service console.
2. Select Stacks on the left navigation pane.
3. Select the Create stack button near the top right side of the page, choosing With new resources (standard) option.
4. Under Specify template, choose Upload a template file option and upload the CloudFormation template named ad-sync.yaml.
5. Select Next.
6. On the Specify stack details page, provide a stack name, along with appropriate values for each parameter. Select Next.
   1. SCIM Synchronization Frequency – As you fine-tune your desired synchronization frequency, invoke the Lambda function multiple times to determine the average execution time. The execution time can vary based on factors such as the number of users and groups to synchronize and network latency. When setting your frequency, consider your business requirements and the Lambda function’s maximum timeout of 15 minutes as well.
   2. Prefixes for AD groups – To specify multiple prefixes, separate them with commas.
   3. AD group search bases – To specify multiple search bases, separate them with semicolons.
   4. AD user search bases – To specify multiple search bases, separate them with semicolons.
   5. Domain controllers – If possible, specify multiple domain controllers for redundancy, separating them with commas.
7. Acknowledge that CloudFormation might create IAM resources at the bottom of the page and select Next.
8. Review all the options you selected and select Submit. It takes several minutes for the stack to be created.

Create and add a Lambda layer

1. After the stack has been created successfully, you can use either AWS CloudShell or a terminal with the AWS Command Line Interface (AWS CLI) and jq installed for subsequent steps.
2. If using CloudShell, upload the requirements.txt file to a directory in CloudShell. If you chose to enforce TLS on the connection to AD, upload the CA certs file to the same directory as well.
3. Run the following commands in the directory, replacing the placeholders with actual values.
   1. If uploading both requirements.txt file and CA certs file, run the following commands.

   stack_name="<stack_name_placeholder>"
   ca_certs_file_name="<ca_certs_file_name_placeholder>"
   zip_file_name=$(aws cloudformation describe-stacks --stack-name $stack_name | jq -r '.Stacks[0].Parameters[] | select(.ParameterKey == "pLambdaLayerZipFileName") | .ParameterValue')
   layer_bucket_name=$(aws cloudformation describe-stack-resources --stack-name $stack_name --logical-resource-id ScimSyncLambdaLayerBucket | jq -r '.StackResources[0].PhysicalResourceId')
   zip $zip_file_name requirements.txt $ca_certs_file_name
   aws s3 cp $zip_file_name s3://$layer_bucket_name

   2. If uploading requirements.txt file only, run the following commands.

   stack_name="<stack_name_placeholder>"
   zip_file_name=$(aws cloudformation describe-stacks --stack-name $stack_name | jq -r '.Stacks[0].Parameters[] | select(.ParameterKey == "pLambdaLayerZipFileName") | .ParameterValue')
   layer_bucket_name=$(aws cloudformation describe-stack-resources --stack-name $stack_name --logical-resource-id ScimSyncLambdaLayerBucket | jq -r '.StackResources[0].PhysicalResourceId')
   zip $zip_file_name requirements.txt
   aws s3 cp $zip_file_name s3://$layer_bucket_name

4. Run the following command and navigate to Amazon Simple Storage Service (Amazon S3) console to locate the bucket with the name that matches the command output.

aws cloudformation describe-stack-resources --stack-name $stack_name --logical-resource-id ScimSyncLambdaLayerBucket | jq -r '.StackResources[0].PhysicalResourceId'

5. When you select the bucket name, you should see the zip file that was uploaded by running the commands in Step 3 (Figure 2).

Figure 2 – Lambda layer zip file uploaded to Amazon S3 bucket

6. Run the following command and navigate to AWS CodePipeline console to locate the pipeline with the name that matches the command output from the following.

aws cloudformation describe-stack-resources --stack-name $stack_name --logical-resource-id ScimSyncCodePipeline | jq -r '.StackResources[0].PhysicalResourceId'

7. When you select the pipeline name, you should see both stages succeeded in the pipeline that was started when you uploaded the zip file in Step 3 (Figure 3).

Figure 3 – Pipeline with both stages succeeded

Invoke the Lambda function

1. Before invoking the function, consider the following.
   1. Note the attribute mappings between AD and IAM Identity Store (Figure 4).
      

      Figure 4 – Attribute mappings between Active Directory and IAM Identity Center

   2. Ensure that AD users have the required attributes (mail, displayName, sn, and givenName) populated. Users without those attributes will not be provisioned into IAM Identity Center.
   3. Nested groups are NOT supported by this solution.
   4. Existing users in IAM Identity Center are always updated even if there are no changes to their attributes.
   5. Manually created users (or users with no or non-matching externalId values) in IAM Identity Center will be deleted during synchronization.
   6. If there are multiple AD users with the same mail attribute value, only one of them will be provisioned because userName must be unique in IAM Identity Center.
   7. Unexpected characters in AD user attributes (e.g., leading/trailing spaces in email addresses) can cause issues with the IAM Identity Center SCIM protocol.

2. Run the following command and navigate to the Lambda console to locate the Lambda function with the name that matches the command output.

aws cloudformation describe-stack-resources --stack-name $stack_name --logical-resource-id ScimSyncLambdaFunction | jq -r '.StackResources[0].PhysicalResourceId'

3. Select the Lambda function name, then on the Test tab. Invoke the function by selecting Test (Figure 5).

Figure 5 – Test button to manually invoke the Lambda function

4. Review the logs inside the CloudWatch Logs log group associated with the Lambda function by selecting the logs link (Figure 6).

Figure 6 – Link to view logs after the Lambda function completes execution

5. Navigate to IAM Identity Center and verify that users and groups from AD have been provisioned.

[Optional] Set up custom SCIM token expiration notification

1. If you enabled custom SCIM token expiration notification when you created the stack in the member account, run the following command and save the output for Step 7.

aws cloudformation describe-stacks --stack-name $stack_name | jq -r '.Stacks[0].Outputs[] | select(.OutputKey == "ScimTokenExpirationNotificationEventBusArn") | .OutputValue'

2. Log into the management account and navigate to CloudFormation service console in the same AWS Region as in the member account.
3. Select Stacks on the left navigation pane.
4. Select the Create stack button near the top right side of the page, choosing With new resources (standard) option.
5. Under Specify template, choose Upload a template file option and upload the CloudFormation template named ad-sync-scim-cross-account-event-notification.yaml. Select Next.
6. On the Specify stack details page, provide a stack name, along with appropriate values for each parameter (Figure 7). For the parameter named Cross Account SCIM Token Expiration Event Bus ARN, paste the value saved from Step 1. Then select Next.

Figure 7 – Providing parameter values for the CloudFormation template

7. Acknowledge that CloudFormation might create IAM resources at the bottom of the page and select Next.
8. Review all the options you selected and select Submit. It will take a couple of minutes for the stack to be created.

Cleanup

To avoid incurring future charges, delete the following resources.

1. In the member account:
   1. Navigate to CloudFormation service console and delete the stack you created earlier. This deletion could take several minutes.
   2. If deletion fails due to non-empty Amazon S3 buckets, empty them via Amazon S3 service console and retry the deletion.
2. If you deployed a stack in the management account, navigate to CloudFormation service console and delete the stack you created there.

Conclusion

In this blog, we provided a solution to automate the process of provisioning users and groups from AD into IAM Identity Center using the SCIM protocol, ensuring consistent identity management to provide federated access to AWS accounts and resources. By reducing manual effort and potential errors, this tool can be a valuable asset for organizations that rely on both AD and IAM Identity Center for their identity and access management needs.

AWS has significantly more services, and more features within those services, than any other cloud provider, making it faster, easier, and more cost effective to move your existing applications to the cloud and build nearly anything you can imagine. Give your Microsoft applications the infrastructure they need to drive the business outcomes you want. Visit our .NET on AWS and AWS Database blogs for additional guidance and options for your Microsoft workloads. Contact us to start your migration and modernization journey today.