How to Move More Custom Applications to the AWS Cloud with AWS Directory Service

Some Active Directory (AD) integrated applications require custom changes to the directory schema. Today, we have added the ability for an administrator to extend the schema of AWS Directory Service for Microsoft Active Directory (Enterprise Edition), also known as Microsoft AD. Specifically, you can modify the AD schema and enable many more applications. This feature also allows you to add new attributes and object classes to your AD that your application requires and that are not present in the core AD classes and attributes. Finally, it allows you to rename and disable attributes you create.

To update your schema, you upload a compatible Lightweight Directory Access Protocol Data Interchange Format (LDIF) file through the Directory Service console or AWS SDK. LDIF is a standard for formatted text designed to exchange data and update schemas for Lightweight Directory Access Protocol (LDAP) servers such as AD. Applications that require elevated permissions, such as Enterprise or Domain Admins, might not be supported.

In this blog post, I explain schema attributes and classes, and I give an overview of LDIF files and formatting. I then walk through a use case, which adds a new attribute to the computer class object that stores the Amazon EC2 instance identifier for my EC2 instances that are joined to my Microsoft AD domain, in three main steps:

1. Create an LDIF file.
2. Import an LDIF file.
3. Validate schema updates.

I also show how to add a value to the new attribute.

Schema concepts

Schemas define the structures of directories and are composed of object classes that contain attributes, which can be uniquely referenced by an object identifier. Before you can modify a schema, it is useful to know how schemas are defined. This section covers important concepts you need to know about AD schemas. If you are already familiar with AD schemas and LDIF files, feel free to skip ahead. Note: All links in this section go to the Microsoft Developer Network (MSDN) website.

Schema classes: Each schema class, similar to a table in a database, has several properties, such as objectClassCategory, that define the class category. Go to Characteristics of Object Classes to see the complete list of classes’ characteristics, and learn more about how to create a new class at Defining a New Class.

Schema attributes: Each schema attribute, which is similar to a field in a database, has several properties that define its characteristics. For example, the property used by LDAP clients to read and write the attribute is the IDAPDDisplayName property, which must be unique across all attributes and classes. Characteristics of Attributes has a complete list of attribute characteristics; you can find additional guidance for creating a new attribute on Defining a New Attribute.

Schema linked attributes: Some attributes are linked between two classes with forward and back links. For example, when you add a user to a group, AD creates a forward link to the group, and AD adds a back link from the group to the user. A unique linkID must be generated when creating an attribute that will be linked.

Object identifier (OID): Each class and attribute must have an OID that is unique for all your objects. Software vendors must obtain their own unique OID to ensure uniqueness to avoid conflicts when more than one application uses the same attribute for different purposes. To ensure uniqueness, you can obtain a root OID from an ISO Name Registration Authority. Alternatively, you can obtain a base OID from Microsoft. To learn more about OIDs and how to obtain them, go to Object Identifiers.

LDIF files and formatting

LDIF files are formatted text files that you can use to modify objects and schemas in an LDAP directory such as AD. LDIF files contain instructions to define classes and attributes for objects that are stored in the directory. An instruction to define a class or an attribute is composed of multiple lines, each specifying a different property of the class or attribute. The format for LDIF files is an Internet Engineering Task Force (IETF) standard defined in Request for Comments (RFC) 2849.

To modify the schema in your Microsoft AD directory, you must first obtain or create an LDIF file. You must also have permissions to modify any objects held within the organizational unit (OU) that your Microsoft AD administrative account is delegated to control. You can also obtain an LDIF file by exporting one from a preexisting directory. Often, software manufacturers will provide a pre-created LDIF file with their products. When you have your LDIF file, you submit the LDIF file to extend the existing schema.

LDAP directories do not read the LDIF file. Rather, a program (for example, Ldifde.exe) interprets the LDIF file. The program converts the LDIF instructions into a sequence of LDAP commands that it sends to the directory, using the LDAP protocol. In the case of Microsoft AD, the LDIF file is submitted through the Directory Service console or AWS SDK; you do not have permissions to make modifications to the schema directly from an LDAP tool or application running in your Amazon VPC.

Important: Schema modifications are a critical operation, and you must perform them with extreme caution. LDIF file errors can break your applications! To recover from an error, you can only apply another modification to disable changes you made, or restore your entire directory to a previous state, which results in directory down time. Carefully plan, approve, and test all schema changes in a test environment first. Please review the AWS Shared Responsibility Model.

Whether you create an LDIF file from the ground up, export an LDIF file from another AD schema, or use an LDIF file supplied by your software vendor, you must understand a few key LDIF file formatting rules. In all cases, you must be familiar with the standards to modify the file when required. For example, each modification in an LDIF file is preceded by a distinguished name (DN) that uniquely identifies the object you are changing in the schema. The DNs in your LDIF file must exactly match the DNs of your directory. If you import an LDIF from a different directory, the DNs must be edited throughout the LDIF file. See the LDIF formatting rules in the Extending Your Microsoft AD Schema.

AD operates from a directory schema stored in a cache that is filled from the directory stored on disk. AD refreshes the cache every 5 minutes, and when schema changes are made, they are applied only to the directory stored on disk. If you need the schema changes stored on disk to take effect immediately, you must issue a command to update the schema cache after making each change.

How to extend your Microsoft AD directory schema

In the remainder of this blog post, I show how to extend your Microsoft AD directory schema by adding a new attribute to the computer object class to store the EC2 instance identifier. This walkthrough follows three main steps: 1) Create an LDIF file, 2) import an LDIF file, and 3) Validate schema updates. For this demonstration, my company name is Example, Inc., and my directory uses the domain name, example.com.

Step 1: Create an LDIF file

To create your own LDIF file:

1. Define your new schema attribute.
2. Define your new schema object class.
3. Create the LDIF file.

A. Define your new schema attribute

In this example, the new attribute is named EC2InstanceID. To make my attribute name unique within the schema, I add example- as a prefix. The prefix and name act as a form of documentation about the creator and purpose of an object when an administrator is browsing the directory schema. In this case, the common name (CN) and IDAPDisplayName are the same: example-EC2InstanceID. To define your own prefix for your schema changes, you can use your DNS, an acronym, or another string that is unique to your company. For now, I am defining my attribute and property values that I will use later when I create the LDIF file in Step 3.

The following table shows the complete set of properties for creating my new attribute. To learn more details about attributes and properties, go to Defining a New Attribute and System Checks and Restrictions Imposed on Schema Additions and Modifications.

Note: Because this attribute does not have an oMSyntax of 127, an oMObjectClass is not called for, in this example.

B. Define your new schema object class

AD comes with a core schema that defines objects used by the Windows operating system and many AD-integrated applications. You should not modify core schema definitions for AD, also called Category 1 objects. Instead, you should create a new auxiliary class and add the new attributes to this new auxiliary class. Then, add the attributes defined in the new auxiliary class to the core class to extend the schema so that the directory remains compatible with applications that depend on the core schema.

In my example, my goal is to add the example-EC2InstanceID attribute to the Computer class. To do so, I create a new auxiliary class named example-Computer and add the example-EC2InstanceID attribute to this new auxiliary class. Later, I will add the example-EC2InstanceID attribute defined in the example-Computer auxiliary class to the Computer core class. For now, I define only the example-Computer class that I will translate into LDIF instructions in Step 3.

The following table shows the complete set of properties for creating my new auxiliary class. To learn more details about classes and properties, go to Defining a New Class.

C. Create the LDIF file

Now that I have defined my new example-EC2InstanceID attribute and example-Computer auxiliary class, it is time to put everything together in a single LDIF file. The sequence of LDIF instructions to extend the schema in my example is:

1. Define the example-EC2InstanceID.
2. Update the schema cache.
3. Define the example-Computer auxiliary class.
4. Update schema cache.
5. Add the attributes defined in the example-Computer auxiliary class to the Computer.
6. Update schema cache.

For each new instruction in the LDIF file, you must define the distinguished name (DN) as the first line of the instruction. The DN identifies an AD object within the AD object’s tree and must contain the domain components for your directory. In my example, the domain components for my directory are DC=example,DC=com.

The DN also must contain the common name (CN) of the AD object. The first CN entry is the attribute or class name. Next, you must use CN=Schema,CN=Configuration. This CN ensures that you can extend the AD schema. As mentioned before, you cannot add or modify AD objects’ content.

Remember: If you are creating a new file, using a file created by an application provider, or using a file exported from another domain, you must update the DNs throughout the LDIF file to match your directory’s naming conventions. The general format for a DN follows.

dn: CN=[attribute or class name],CN=Schema,CN=Configuration,DC=[domain_name]

In my example, the DN for my new attribute example-EC2InstanceID follows.

dn: CN=example-EC2InstanceID,CN=Schema,CN=Configuration,DC=example,DC=com

I create an LDIF file by using the instructions for the six previously named steps to define and add my new attribute to the AD schema. I name my LDIF file schema_EC2InstanceID.ldif, which is available for download. If you want to perform a test with the file, don’t forget to change the dn entries to specify your own domain name. The following is the code of the LDIF file.

# 1 – Define the example-EC2InstanceID attribute

dn: CN=example-EC2InstanceID,CN=Schema,CN=Configuration,DC=example,DC=com
changetype: add
objectClass: top
objectClass: attributeSchema
cn: example-EC2InstanceID
attributeID: 1.2.840.113556.1.8000.9999.2.1
attributeSyntax: 2.5.5.12
isSingleValued: TRUE
adminDisplayName: example-EC2InstanceID
adminDescription: EC2 Instance ID
oMSyntax: 64
searchFlags: 1
lDAPDisplayName: example-EC2InstanceID
systemOnly: FALSE

# 2 – Update the schema cache

dn:
changetype: modify
add: schemaUpdateNow
schemaUpdateNow: 1
-

# 3 – Define the example-Computer auxiliary class

dn: CN=example-Computer,CN=Schema,CN=Configuration,DC=example,DC=com
changetype: add
objectClass: top
objectClass: classSchema
cn: example-Computer
governsID: 1.2.840.113556.1.8000.9999.1.1
mayContain: example-EC2InstanceID
rDNAttID: cn
adminDisplayName: example-Computer
adminDescription:  Example Computer Class
objectClassCategory: 3
lDAPDisplayName: example-Computer
name: example-Computer
systemOnly: FALSE

# 4 – Update the schema cache

dn:
changetype: modify
add: schemaUpdateNow
schemaUpdateNow: 1
-

# 5 – Add the attributes defined in the example-Computer auxiliary class to the

# Computer class

dn: CN=Computer,CN=Schema,CN=Configuration,DC=example,DC=com
changetype: modify
add: auxiliaryClass
auxiliaryClass: example-Computer
-

# 6 – Update the schema cache

dn:
changetype: modify
add: schemaUpdateNow
schemaUpdateNow: 1
-

Step 2: Import an LDIF file

After carefully reviewing and saving my LDIF file, I am now ready to upload it and apply it to my directory:

1. Open the AWS Management Console, choose Directory Service, and then choose your Microsoft AD Directory ID link (as shown in the following screenshot).
   
2. Choose the Schema extensions tab. Here you can see a history of all the schema modification attempts made to the directory with the Start Time, End Time, and Description of the update provided by you, and the Status.
   

3. Choose Upload and update schema to upload your LDIF file.
4. On the Upload LDIF file and update schema page, choose Browse to choose your LDIF file. Type a description for this schema update to document why you updated your schema, and choose Update Schema.
   

When you choose Update Schema, Microsoft AD performs an initial syntax validation of your LDIF file.

Note: The most common syntax validation error is failing to update the DNs in your LDIF file to match your directory’s naming conventions.

After the initial LDIF file syntax validation, three actions are performed automatically:

1. Microsoft AD creates a snapshot. The snapshot allows you to restore your directory just in case you face any issues in your application after updating the schema. The creation of the snapshot takes approximately 40 minutes. If you have reached your manual snapshot limit, Microsoft AD will prompt you to delete one of your manual snapshots before continuing.
2. Your LDIF file is applied against your directory to a domain controller in isolation. Microsoft AD selects one of your DCs to be the schema master. It removes that DC from directory replication and applies your LDIF file using Ldifde.exe. This operation usually takes less than a minute.
3. The schema updates are replicated to all domain controllers. After successfully applying your LDIF file against your domain controller in isolation, the modified DC is added back into replication, and the schema updates are replicated to all domain controllers.

While these steps are in progress, your directory is available to respond to application and user requests. However, you cannot perform directory maintenance operations such as creating trusts or adding IP routes. You can monitor the progress in the schema update history, as shown in the next screenshot.

Note: If any errors occur during the update, you receive a detailed error message and, when possible, a line number to show where the error occurred in your LDIF file. If an error occurs, your Microsoft AD schema will remain unmodified.

Step 3: Validate the schema updates

The last step is to validate if your schema updates have been applied to your directory. This is a key step before you migrate or update any application that relies on the schema updates. You can accomplish this by using a variety of different LDAP tools, or by writing a test tool that issues appropriate LDAP commands. I will show you two ways to accomplish this: by using the AD Schema Snap-in and Windows PowerShell.

To verify if the schema updates are present, you must run AD tools from a computer that is joined to your Microsoft AD domain. This can be a Windows server running in your on-premises network, with access to your Amazon Virtual Private Cloud (VPC) through a Virtual Private Network (VPN) connection. You also can perform this on an Amazon EC2 Windows instance (see Joining a Domain Using the Amazon EC2 Launch Wizard).

To use the AD Schema Snap-in:

1. Install the AD Schema Snap-In by following this procedure.
2. Open the Microsoft Management Console (MMC) and expand the Active Directory Schema tree for your directory. You should be able to see your classes and attributes. The following screenshot shows the example-EC2InstanceID attribute.
   

If you prefer to use Windows PowerShell, you can use it to see if your schema update was applied to your directory by using the Get-ADObject cmdlet. The following code shows the syntax of my example.

# New attribute EC2InstanceID
PS C:\> get-adobject -Identity 'CN= EC2InstanceID,CN=Schema,CN=Configuration,DC=example,DC=com' -Properties * <Enter>

adminDescription: EC2 instance ID
adminDisplayName: example-EC2InstanceID
attributeID     : 1.2.840.113556.1.8000.9999.2.1
attributeSyntax : 2.5.5.12
CanonicalName   : example.com/Configuration/Schema/example-EC2InstanceID
CN              : example-EC2InstanceID
...

# New auxiliary class example-computer
PS C:\> get-adobject -Identity 'CN=example-Computer,CN=Schema,CN=Configuration,DC=example,DC=com' -Properties * <Enter>

adminDescription: example computer class
adminDisplayName: example-Computer
CanonicalName   : example.com/Configuration/Schema/example-Computer
CN              : example-Computer
...

# New auxiliary class example-computer added to the computer class
PS C:\> get-adobject -Identity 'CN=computer,CN=Schema,CN=Configuration,DC=example,DC=com' -Properties auxiliaryClass | select -ExpandProperty auxiliaryClass <Enter>

example-Computer
ipHost

Optional: Add a value to the new attribute

Now that you have created a new attribute, let’s add a new value to the attribute of a computer in your Microsoft AD directory. Open Windows PowerShell and set the new attribute with the following command.

PS C:\> set-adcomputer -Identity <computer name> -add @{example-EC2InstanceID = '<EC2 instance ID>'}

Now, validate if the added EC2InstanceID value to a computer object in the previous command is present in your Microsoft AD directory:

PS C:\> get-adcomputer -Identity <computer name> –Property example-EC2InstanceID

DistinguishedName     : CN=<computer name>,OU=Computers,DC=example,DC=com
DNSHostName           : <computer name>.example.com
Enabled               : True
example-EC2InstanceID : <EC2 instance ID>
Name                  : <computer name>
ObjectClass           : computer
ObjectGUID            : e7c2f596-4f8c-4084-a124-6fc6babaf3c8
SamAccountName        : <computer name>
SID                   : S-1-5-21-3794764471-3203795520-307520662-1104

Summary

In this blog post, I provided an overview of the key concepts, terms, and steps involved in extending a Microsoft AD schema, including how to format your LDIF file and how to apply your LDIF file against your Microsoft AD directory using the Directory Service console. Even if you are importing the LDIF file from another domain or application, these steps are important to understanding the potential impact on your directory. Extending the schema is a critical operation. Do not apply any schema update in your production environment without testing it with your application in a dev/test environment first.

For additional information about schema extensions, how to create LDIF files, and common errors, see the AWS Directory Service documentation.

To learn more about AWS Directory Service, see the AWS Directory Service home page. If you have questions, post them on the Directory Service forum.

– Peter