Enabling multiprotocol workloads with Amazon FSx for NetApp ONTAP

In an enterprise environment, it is common that both Linux and Windows workloads require access to the same data. For example, electronic design automation (EDA) and media and entertainment workloads often involve Windows users accessing data produced by Linux compute instances. Concurrent access to this data requires multiprotocol access: NFS for Linux workloads and SMB for Windows workloads. Multiprotocol shared storage enables the same dataset to be presented via both Linux and Windows so that each operating system can access the data with its native-file protocol.

With the announcement of Amazon FSx for NetApp ONTAP, AWS provides your Linux, Windows, and macOS workloads with concurrent, multiprotocol access to fully managed shared storage combined with ONTAP’s popular data management capabilities. With FSx for ONTAP, you can consolidate your Linux and Windows workloads into a single service or file system, even if these workloads do not share the same data. This will allow you to reduce overhead and complexity in your environment, save cost by avoiding multiple copies of data, and migrate existing multiprotocol workloads to AWS.

In this blog, we demonstrate how you can set up multiprotocol access to the same FSx for ONTAP volume using both the AWS Management Console and NetApp ONTAP CLI.

Solution overview

• Authentication and authorization concepts
• Environment setup
• Setting up Active Directory (AD)
• Setting up FSx for ONTAP
• Configuring name mapping using the ONTAP CLI
• Accessing data from Windows and Linux instances

Authentication and authorization concepts

There are a few important concepts to introduce when it comes to multi-protocol data access: user identity mapping and security style.

User identity mapping

User identity mapping is the process of mapping a Linux user to a Windows user and vice versa. Organizations typically manage user identities with an identity provider such as AD or another Lightweight Directory Access Protocol (LDAP) to manage shared credentials across many systems. Both Linux and Windows convert the user name to an underlying ID: for Linux, this is the user identifier (UID); for Windows this is the security identifier (SID).

Each FSx for ONTAP file system can have one or more storage virtual machines (SVMs). An SVM is a virtual file server that has endpoints for client access. Each SVM can have a separate AD or other authentication configuration. The configuration will determine how the SVM authenticates users and can include unique user mappings between Linux and Windows identifiers. We will show an example of this later in the blog.

Security style

Security style determines how access control is handled for a given ONTAP volume. FSx for ONTAP supports all security styles that ONTAP supports, including NTFS, UNIX, and mixed (the default is UNIX). The best security style for a given environment depends on how permissions are managed, though any security style can be used for concurrent Windows and Linux access. If any Windows access is required, NTFS is the recommended security style. The “mixed” security style is generally only recommended for advanced use cases when multiple types of permissions need to be managed on the same volume, as permissions will be selected depending on the client accessing data. An individual file or directory uses only one type of security style at a time, so on a given volume it may be the case that directory1 is using UNIX permissions, while directory2 is using NTFS permissions.

Figure 1: Security styles

Note: Security style can be configured at both the SVM and the volume level. The SVM level setting will set it on the root volume and any volumes under that SVM will inherit the setting unless it has its own explicit setting

Environment setup

The environment for this blog consists of an FSx for ONTAP file system, an AWS Managed Microsoft Active Directory (AD), a Windows 2019 Amazon EC2 instance, and an Amazon Linux 2 EC2 instance (Figure 1). It uses Active Directory (AD) for authentication with a single SVM joined to the AD. All resources reside in a single Amazon Virtual Private Cloud (VPC).

Note: If you have resources in separate VPCs, clouds, or on premises, you can use AWS Transit Gateway to connect these resources and enable multiprotocol resource access.

This blog uses the Amazon FSx console and the NetApp ONTAP CLI. To access the ONTAP CLI, you need to SSH into your FSx for ONTAP file system management endpoint (shown later).

Figure 2: Amazon FSx for NetApp ONTAP file system (high-level architecture)

You need to determine which security style to use that best fits your file system’s protocol and preferred file permission management style. The following table indicates which security style to use based on these inputs, as well as how users are mapped between Linux (UNIX) and Windows.

In this blog, we use the NTFS security style, manage file permissions using NTFS ACLs, and map the Linux LDAP user attributes uid, uidNumber, and gidNumber to Windows users in AD. We show how to configure this later in the blog.

Figure 3: Overview of name mapping for your file system

Setting up Active Directory

This blog uses AWS Managed Microsoft AD for managing users (both Windows and Linux), but your own AD solution will work as well. For more information about setting up AWS Managed Microsoft AD, see the AWS documentation.

In Active Directory, we add a user (named “Bob”). This user is for testing access from your Linux and Windows clients. Using the Attribute Editor in Active Directory Users and Computers, we set the following attributes for the user:

uid: bob
uidNumber: 10001
gidNumber: 20001

Note: The uidNumber and gidNumber depends on your environment. These numbers can be any value as long as it does not conflict with other users and is less than the maximum value supported by the operating system version in your environment.

Figure 4: Active Directory user properties

We also create a second user “Mary” to test access restriction:

uid: mary
uidNumber: 10002
gidNumber: 20001

Setting up FSx for ONTAP

In the Amazon FSx console, select Amazon FSx for NetApp ONTAP.

Figure 5: Amazon FSx file system options in Amazon FSx console

Create a file system using Standard create, as the option to join an SVM to Active Directory is only available through standard create.

Figure 6: FSx for NetApp ONTAP file system creation method

In Security & encryption, specify a password. You’ll use this to log into your FSx for ONTAP file system later in the blog.

Figure 7: Security and Encryption

After we specify the basic File system details, we configure a default storage machine configuration. Give it a name, and optionally specify a password (an SVM password is not required for this blog). Then choose Join an Active Directory, which is a critical aspect of this setup. Here, specify the DNS addresses associated to your AD configuration.

In the Active Directory configuration, specify the Organizational Unit (OU) that corresponds to our Domain Controller (DC) and OUs within which to join our file system. For more information on these values, see the AWS documentation.

Figure 8: Active Directory SVM configuration

In Default volume configuration, provide a volume name (we used “volmulti”), junction path (/volmulti), and size (10 GiB).

Figure 9: Volume configuration

Review all the configuration on the Summary page, then select Create.

Figure 10: Amazon FSx console file system summary

The file system can take 30–40 minutes to create. Once it has finished creating, proceed to configuring name mapping using the ONTAP CLI.

Configuring name mapping using the ONTAP CLI

The rest of the configuration process uses the ONTAP CLI. To access the ONTAP CLI, SSH into the management IP of the FSx for ONTAP file system, using “fsxadmin” as the user name, and the password you defined when creating the file system. To get the management IP of your file system, see the AWS documentation.

SSH fsxadmin@<management-IP-address>

We first add an LDAP configuration in order to map Linux UIDs to valid users in Active Directory.

Note: There are multiple ways you can resolve Linux UIDs on ONTAP including LDAP, NIS, and local users. We use the AD properties in this case but you could utilize the method that works best for your environment.

Run the vserver services name-service ldap client create command (documented here) to associate an LDAP client configuration with an SVM. The SVM will use the AD credentials provided at SVM creation to connect to AD for LDAP lookups. The relevant parameters we use are as follows:

• vserver specifies the SVM you’re creating the LDAP configuration for.
• client-config is the name for the LDAP client configuration, we called it “ad”.
• ad-domain is the domain of your AD server (ontapdemo.local).
• base-dn is the base distinguished name (DN), the point where the SVM will search for users. This has the OU and DC values we configured for the SVM at creation, minus OU=Computers.
• schema specifies the schema template the SVM uses when making LDAP queries. We use AD-IDMU.

More information on these commands can be found in the NetApp documentation.

FsxIdxxxxxxxxxxxxxxxxx::> vserver services name-service ldap client create -vserver mysvm -client-config ad -ad-domain ontapdemo.local -base-dn "OU=ontapdemo,DC=ontapdemo,DC=local" -schema AD-IDMU

The next command needs the client-config name set in the previous step to associate with the SVM.

FsxIdxxxxxxxxxxxxxxxxx::> vserver services name-service ldap create -vserver mysvm -client-config ad

Next, we add LDAP as a name service.

FsxIdxxxxxxxxxxxxxxxxx::> vserver services name-service ns-switch modify -vserver mysvm -database passwd,group -sources files,ldap

We then set up a name mapping that will associate any Linux user name to the same user name with the NetBIOS domain name format of domain\username. You can use any regular expression you need here to translate between Linux and Windows users. In this case, our domain name is ontapdemo, so we are translating any Linux user name to ontapdemo\username.

FsxIdxxxxxxxxxxxxxxxxx::> vserver name-mapping create -vserver mysvm -direction UNIX-win -position 1 -pattern (.+) -replacement ontapdemo\\\1

Finally, we set up a CIFS (SMB) share so Windows can access the FSx for ONTAP volume we created. An NFS export is automatically configured by FSx for ONTAP when the volume is created for Linux access. Use the junction path provided when creating your file system in the following command. You can use any name you prefer for share-name.

FsxIdxxxxxxxxxxxxxxxxx::> vserver cifs share create -share-name volmulti -path /volmulti -vserver mysvm

Your file system now can be accessed by both Windows and Linux systems, and your SVM has a method of mapping Active Directory users to Linux users.

Multiprotocol data access

We want to make sure we can allow (and restrict) data access for both Windows and Linux users. We log into the Windows system as the user we created in Active Directory that will be allowed to access files (ontapdemo\bob) and navigate to \\mysvm.ontapdemo.local\volmulti in file explorer.

Figure 11: SMB Share

Then, we create a file called test.txt and save some data in it.

Figure 12: Text file for testing data access

We only want “bob” to be able to access this file so we update the Windows NTFS ACL to give only “bob” full access permissions.

Figure 13: File NTFS ACL

For the Linux instance, we can join the AD domain using the AWS documentation. However, we need to take one additional step and set the ldap_id_mapping = False in the sssd.conf file. This enables us to use the uidNumber we set in the users property instead of the generated one from System Security Services Daemon (SSSD). After joining the domain, we run the following commands to make this change.

[ec2-user]$ sudo sed -i 's/ldap_id_mapping = True/ldap_id_mapping = False/I' /etc/sssd/sssd.conf

[ec2-user]$ sudo sss_cache -E

[ec2-user]$ sudo systemctl restart sssd.service

We then log in to our Linux system as “bob” and mount the FSx for ONTAP volume to /mnt/volmulti.

[bob]$sudo mkdir /mnt/volmulti

[bob]$sudo mount -t nfs mysvm.ontapdemo.local:/volmulti /mnt/volmulti

We verify that bob has the correct UID.

[bob]$ id

uid=10001(bob) gid=20001(FSXUser) groups=20001(FSXUser)

We then read the contents of the file created from the Windows system, and see that “bob” shows up as the owner of the file.

[bob]$ cat /mnt/volmulti/test.txt

Here is my multiprotocol file

[bob]$ ls -o test.txt

-rwxrwxrwx 1 bob 27 Aug 18 21:34 test.txt

[bob]$ ls -on test.txt

-rwxrwxrwx 1 10001 27 Aug 18 21:34 test.txt

To verify that we can restrict access for users, we next log in to the Linux system as “Mary” and verify we cannot access the file on Linux based on the NTFS ACL.

[mary]$ id

uid=10002(mary) gid=20002(Users) groups=20002(Users)

[mary]$ cat /mnt/volmulti/test.txt

cat: /mnt/volmulti/test.txt: Permission denied

Even using root permissions, the file cannot be accessed, because even root is mapped to a valid Windows user based on default ONTAP settings, and ONTAP evaluates the NTFS permission based on the user for additional security.

[mary]$ sudo cat /mnt/volmulti/test.txt

cat: /mnt/volmulti/test.txt: Permission denied

We verified multiprotocol access with “bob,” and that neither “mary” (nor “root”) was unable to access the file.

Cleaning up

If you followed the steps in this blog for testing and no longer need the environment, make sure to clean up the resources that were deployed. As part of this blog, you may have created an Amazon FSx for ONTAP file system, an AWS Managed Microsoft Active Directory domain, and two Amazon EC2 instances. To remove the EC2 instances you can use the EC2 console. To remove the Amazon FSx for ONTAP file system you can use the FSx console. To remove the AWS Managed Microsoft Active Directory domain you can use the Directory Service console.

Conclusion

In this blog, we walked you through how to enable native multiprotocol workloads with Amazon FSx for NetApp ONTAP. We showed how to use AWS Managed AD service to enable both Linux and Windows Amazon EC2 instances to access the same FSx for ONTAP volume and data, and that permissions propagated between Linux and Windows using Active Directory. There are many other use cases, and the same basic steps would apply if you are using a self-managed Active Directory setup or another directory service. It is also important to consider how you want to manage access so you can pick the correct security style (NTFS, UNIX, or mixed).

This capability will allow you to simplify your environment instead of having to maintain separate storage for NFS and SMB. It avoids the need to have to figure out how to replicate data between solutions or build complicated workflow logic to move or copy specific data. This will also save you cost by avoiding copying data to two different locations in addition to Amazon FSx for ONTAP built in cost optimization features.

Thanks for reading this blog post on enabling native multiprotocol workloads. If you have any comments or questions, don’t hesitate to leave them in the comments section.