FSI Services Spotlight: Featuring Amazon FSx for NetApp ONTAP

In this edition of the Financial Services Industry (FSI) Services Spotlight monthly blog series, we highlight five key considerations for customers running workloads on Amazon FSx for NetApp ONTAP (FSx for ONTAP) achieving compliance, data protection, isolation of compute environments, audits with APIs, and access control/security. Across each area, we will examine specific guidance, suggested reference architectures, and technical code to help streamline service approval of FSx for ONTAP.

Amazon FSx for NetApp ONTAP is a fully managed file service built on NetApp ONTAP that provides reliable, scalable, and performant shared storage. It is also the first multi-protocol file service available on AWS supporting SMB, NFS, and iSCSI. As a fully managed service, you no longer have to worry about setting up and provisioning file servers and storage volumes, replicating data, installing and patching software, detecting and addressing hardware failures, managing failover and failback, and manually performing backups.

FSx for ONTAP can serve a wide range of use cases in cloud and hybrid cloud scenarios. It can be optimized for different use cases through configuration. For example, FSx for ONTAP can be used for high-performance storage use cases suitable for use with Oracle, SAP, VMware, and Microsoft SQL Server, using high-performance SSD storage with sub-millisecond latencies. FSx for ONTAP can also cost-optimize general NAS-based workloads by using a fully elastic capacity tier of storage that provides virtually unlimited, lower cost storage.

FSx for ONTAP works natively with other NetApp products and can serve key roles in hybrid cloud architectures. Additional features that make data management cheaper and easier including inline data compression, deduplication, compaction, thin provisioning, replication (SnapMirror), and point-in-time cloning (FlexClone). For customers that want to maintain their existing technology stacks on-prem and in the cloud, VMware Cloud + FSx for ONTAP combined can speed up implementation of use cases like migration, disaster recovery, or cloud bursting. FSx for ONTAP also has rich integration with other AWS services, AWS Identity and Access Management (IAM), Amazon WorkSpaces, Key Management Service (KMS), AWS CloudTrail, and the general compute services EC2, ECS, and EKS.

Achieving compliance with FSx for NetApp ONTAP

Achieving compliance is made easier by inheriting standards of the Amazon FSx managed service. To fully meet the compliance standards, the customer has a responsibility to configure and maintain their environment. Security and compliance are a shared responsibility between AWS and the customer. Compared to deploying NetApp Cloud Volumes ONTAP, which is not a managed service, customers are responsible for fewer controls to deploy a compliant and secure filesystem. Customers should determine their requirements for network connectivity, encryption, and access control in order to properly configure the environment and service. This article will provide additional guidance on security configurations that are part of the customer responsibilities.

The AWS services compliance page has all the compliance programs that Amazon FSx is approved for. As a shorter list for FSI readers, Amazon FSx compliance includes:

• SOC 1,2,3
• PCI
• ISO/IEC 27001:2013, 27017:2015, 27018:2019, and ISO/IEC 9001:2015
• OSPAR
• FINMA
• IRAP

Data Protection

Encryption at Rest

FSx for ONTAP enables encryption at rest by default and uses an AES-256 encryption algorithm for data and metadata at rest. Data is automatically encrypted before being written to disk, and automatically decrypted as it is read. Encryption keys are managed and integrated with FSx using AWS Key Management Service (KMS). The AWS key management infrastructure uses Federal Information Processing Standards (FIPS) 140-2 approved cryptographic algorithms. The infrastructure is consistent with National Institute of Standards and Technology (NIST) 800-57 recommendations.

Figure 1: Select encryption key during file system creation

As seen in Figure 1, when setting up your file system, there is a default KMS key preselected and used for encryption at rest. You can instead choose to specify your own KMS key. NetApp Volume Encryption (NVE), which is used on premise, is not needed to achieve encryption at rest in the cloud.

Encryption in Transit

Understanding how FSx for ONTAP communicates and for what purpose is important in securing all transit protocols. SMB, NFS, and iSCSI are used by clients for file access. HTTPS is for the AWS Console, AWS API, and ONTAP REST API. SSH is used for administrative access to the ONTAP CLI.

Figure 2: FSx for ONTAP multi-AZ architecture

Figure 2 illustrates how a filesystem spans AWS Availability Zones. The inter-node communication used to support high availability and data replication between node pairs uses TLS 1.2. You can configure inter-cluster communication using TLS or IPSEC (TLS is recommended for performance).

For end user file share access, NFS is enabled by default with no encryption or authentication. Encryption in transit is supported with SMB protocol 3.0 or newer, but SMB shares and SMB encryption are not enabled by default. To enabled SMB shares please read the article Enabling multiprotocol workloads with Amazon FSx for NetApp ONTAP. Once enabled, FSx for ONTAP automatically encrypts data in transit using SMB encryption as you access your file system. You can either enable SMB encryption on individual shares, or on a SVM, which turns it on for all shares on that SVM. FSx for ONTAP supports 128 or 256 bit, but the level of encryption used is determined by the client. See detailed instructions for enabling SMB encryption in our user guide.

(Note: the SVM name “fsx” is the default name for your first auto-created SVM and is used in code samples throughout the article for the “-vserver” argument. You can replace “fsx” with the name of your SVM when appropriate)

ONTAP CLI: enabling SMB encryption for all shares of a SVM

 vserver cifs security modify -vserver fsx -is-smb-encryption-required true

ONTAP uses SMB signing protects the security of the data fabric by making sure that traffic between storage systems and clients is not compromised by replay or man-in-the-middle attacks. It does so by verifying that SMB messages have valid signatures. This feature is not enabled by default, you can enable it using the ONTAP CLI using the below command.

ONTAP CLI:

vserver cifs security modify -vserver fsx -kerberos-clock-skew 3 -
kerberos-ticket-age 8 -is-signing-required true

FSx for ONTAP has multiple versions of SMB and NFS enabled by default (SMB 2&3, NFS 3&4). You can disable unwanted version using the ONTAP cli.

ONTAP CLI: disabling smb1, smb2, nfs3, and nfs4

set -privilege advanced
vserver cifs options modify -vserver fsx -smb1-enabled false
vserver cifs options modify -vserver fsx -smb2-enabled false
vserver nfs modify -vserver fsx -v3 disabled
vserver nfs modify -vserver fsx -v4 disabled

Antivirus

For customers who require centralized protection, FSx for ONTAP supports the NetApp virus scanning feature, Vscan. When combined with additionally purchased partner antivirus applications from Symantec, Mcafee, or Trend Micro, and Sophos FSx for ONTAP can automatically scan new files as they’re written to your file system. Antivirus software used for Vscan is purchased separately and deployed separately. Further information can be found in the NetApp documentation for Vscan.

Isolation of Compute Environments

A file system is the primary resource in Amazon FSx for NetApp ONTAP (analogous to an ONTAP cluster on-premises) deployed within an Amazon Virtual Private Cloud (VPC) and subnets. Each file system has a management endpoint that you can use to manage your data using the ONTAP CLI or ONTAP REST API, and an inter-cluster endpoint for replication or caching configurations. Each file system has a VM-enforced isolation boundary and does not share resources with another file system.

As seen in Figure 3, a storage virtual machine (SVM) is a logically isolated file server with its own administrative credentials and network endpoints for administering and accessing data. A default SVM named “fsx” is created when you create a file system using the AWS Console. Each SVM has unique authentication credentials and network access controls. A SVM has up to 4 network endpoints, 3 to access data (NFS, SMB, and ISCSI) and 1 to manage the SVM.

Figure 3: FSx for ONTAP endpoints and isolation boundaries

For network access controls, AWS Security Groups and Network ACLs are attach to SVM endpoints and subnets. A security group acts as a virtual firewall for your FSx for ONTAP file systems to control incoming and outgoing traffic. Since FSx ONTAP can be configured with different features, you should review ports used for your use cases and enable as needed. The following list includes common ports needed in a basic configuration, but you can see a complete list of ports here.

View more

Access from Peered Networks

FSx for ONTAP SVM’s exposes 4 endpoints within your private VPC. When using Single-AZ mode, all endpoint IPs are within the same CIDR (Classless Inter-Domain Routing) of the subnet you deployed to. When used in Multi-AZ mode, the Management, NFS and SMB endpoints use a floating IP range (198.18.0.0/16 by default) which is outside of your VPC CIDR. Clients connecting to these endpoints from outside the VPC, need to use a connection method that supports transitive routing. This includes both AWS Transit Gateway (TGW) and AWS VPN. You can find these IP addresses in the console view of the SVM as pictured in Figure 4.

Figure 4: SVM Fixed IP and Floating IP endpoints

ISCSI endpoints and the filesystem inter-cluster endpoint will use IP addresses from the VPC CIDR. This means that non-transitive connection methods, such as VPC Peering, can be used in addition to transitive methods for these use cases. Figure 5 depicts a common pattern for syncing a FSx filesystem with an on prem NetApp device.

Figure 5: FSx for ONTAP syncing with on prem NetApp units

The inter-cluster endpoints can also be used to support cross region replication, FlexCache, or GlobalFileCache.

Automating Audits with APIs

FSx for ONTAP has 3 types of audit trails to consider:

1. Administrative FSx service API calls
2. Administrative ONTAP CLI commands
3. End user file share access

Administrative FSx service API calls

AWS CloudTrail is a service that can log API requests to the FSx service. It is useful for tracking administrative events like the deleting of a volume, SVM, or file system. CloudTrail cannot track file or folder level events and it is not enabled by default. Learn more on how to setup CloudTrail.

Some key APIs to monitor would be the CreateFileSystem and DeleteFileSystem calls.

Here is an example of a CreateFileSystem CloudTrail log entry, keep in mind the same API is used for different FSX filesystem types, in this case the attribute “fileSystemType” is ONTAP.

{
    "eventVersion": "1.08",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AROAAAAABBBBAAAABBBBA:example",
        "arn": "arn:aws:sts::121212121212:assumed-role/example",
        "accountId": "327468301555",
        "accessKeyId": "ASIAAAAABBBBAAAABBBBA",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "AROAAAAAABBBBAAAABBBBA",
                "arn": "arn:aws:iam::121212121212:role/example",
                "accountId": "121212121212",
                "userName": "example"
            },
            "webIdFederationData": {},
            "attributes": {
                "creationDate": "2022-03-15T13:35:49Z",
                "mfaAuthenticated": "true"
            }
        }
    },
    "eventTime": "2022-03-15T14:26:58Z",
    "eventSource": "fsx.amazonaws.com",
    "eventName": "CreateFileSystem",
    "awsRegion": "us-east-2",
    "sourceIPAddress": "AWS Internal",
    "userAgent": "AWS Internal",
    "requestParameters": {
        "clientRequestToken": "f9c77684-0394-111d-a23e-12121212",
        "fileSystemType": "ONTAP",
        "storageCapacity": 1024,
        "subnetIds": [
            "subnet-0101example",
            "subnet-0101example"
        ],
        "securityGroupIds": [
            "sg-0101example",
            "sg-0101example"
        ],
        "kmsKeyId": "arn:aws:kms:us-east-2:121212121212:key/4ccexex-example",
        "ontapConfiguration": {
            "automaticBackupRetentionDays": 7,
            "deploymentType": "MULTI_AZ_1",
            "diskIopsConfiguration": {
                "mode": "AUTOMATIC"
            },
            "preferredSubnetId": " subnet-0101example ",
            "routeTableIds": [
                "rtb-0101example"
            ],
            "throughputCapacity": 128
        }
    },
    "responseElements": {
        "fileSystem": {
            "ownerId": "121212121212",
            "creationTime": "Mar 15, 2022 2:26:57 PM",
            "fileSystemId": "fs-1212example",
            "fileSystemType": "ONTAP",
            "lifecycle": "CREATING",
            "storageCapacity": 1024,
            "storageType": "SSD",
            "vpcId": "vpc-0101example",
            "subnetIds": [
                "subnet-0101example",
                "subnet-0101example"
            ],
            "kmsKeyId": "arn:aws:kms:us-east-2:121212121212:key/4ccexex-example ",
            "resourceARN": "arn:aws:fsx:us-east-2:121212121212:file-system/ fs-1212example",
            "ontapConfiguration": {
                "automaticBackupRetentionDays": 7,
                "dailyAutomaticBackupStartTime": "05:00",
                "deploymentType": "MULTI_AZ_1",
                "endpointIpAddressRange": "198.19.255.0/24",
                "endpoints": {
                    "intercluster": {
                        "dNSName": "intercluster.fs-1212example.fsx.us-east-2.amazonaws.com"
                    },
                    "management": {
                        "dNSName": "management.fs-1212example.fsx.us-east-2.amazonaws.com"
                    }
                },
                "diskIopsConfiguration": {
                    "mode": "AUTOMATIC",
                    "iops": 3072
                },
                "preferredSubnetId": "subnet-0101example",
                "routeTableIds": [
                    "rtb-0101example"
                ],
                "throughputCapacity": 128,
                "weeklyMaintenanceStartTime": "4:04:00"
            }
        }
    },
    "requestID": "d32e1da5-848e-4d44-84b4-d58b7121212",
    "eventID": "659358ca-1ba7-482c-8f96-40cc7121212",
    "readOnly": false,
    "eventType": "AwsApiCall",
    "apiVersion": "2018-03-01",
    "managementEvent": true,
    "recipientAccountId": "121212121212",
    "eventCategory": "Management",
    "sessionCredentialFromConsole": "true"
}

ONTAP CLI commands

ONTAP CLI logging is enabled by default. When you login with ssh to the filesystem’s or SVM’s management endpoint, you are connecting to the ONTAP CLI. ONTAP CLI command. Audit logs are stored per SVM. History can be viewed using the command “security audit log show”. FSx for ONTAP itself can not automatically monitor and alert based on security audit log entries therefore log data must be moved to a system with those capabilities. ONTAP can integrate with your existing Security Information and Event Management (SIEM) system by forwarding audit logs to any destination supporting syslog. Please read the NetApp documentation on the log forwarding process and how to enable TLS encryption in transit while forwarding logs.

The example security audit log below will contain audit entries for both access and operational tasks:

End user file share access

FSx for ONTAP supports file access auditing requirements by having the ability to track and log events like end-user access to files and directories. This is not enabled by default. Some examples of events that can be logged are successful or failed attempts at logins, file access read/writes, and permission changes. There are different events for SMB and NFS access. You must create an auditing configuration on the SVM, and you must enable it on the SVM. Event logs are stored on the file system in a special folder in EVXT or XML format. Logs can be viewed with windows event viewer.

The following commands on the ONTAP CLI would create rotating logs on a SVM call “fsx” into files up 200MB with a limit of 10 files before older files are overwritten. Staging space for logs is limited to 2GB so enabling rotation is important.

ONTAP CLI: setup file access audit logging with log rotation

vserver audit create -vserver fsx -destination /audit_log -rotate-limit 9 -rotate-size 200M
vserver audit enable -vserver fsx

These file access audit logs are stored locally and cannot be forwarded to a syslog server like the security audit logs can the. However, you can extend the logging solution using additional tools beyond the scope of this article.

Figure 6: System architecture of expanded logging solution

The blog article How to securely share application log files with third parties. Explains the solution depicted in Figure 6. This is useful for common use cases such as log archival, notifications, and ingestion into log management platforms.

Operational Access and Security

Understanding the authentication and authorization mechanisms of FSx for ONTAP will help you help you manage access security

FSx Service

• AWS Console – (IAM)
• Amazon FSx Service API – (IAM)

FSx for ONTAP File System

• Management endpoint – (SSH)
• Inter-cluster endpoint – (pre-shared password)

FSx for ONTAP SVM

• Management endpoint – (SSH)
• NFS endpoint – (NFS none / SMB Kerberos)
• iSCI endpoint – (none)

The AWS Console and Amazon FSx API are used for administrative tasks like backups, creating or deleting file systems and resetting SVM admin passwords.

The FSx for ONTAP File System service account (the cluster administrator in NetApp terms) has access to administer all SVMs under a file system. You can change the password of the fsxadmin account

SVMs, which are isolated file servers within the file system, have their own administrator credential. The SVM management endpoint supports the ONTAP CLI and the ONTAP REST API. The SVM admin has privileges only within its own SVM.

End user access files through the SVM’s NFS endpoint has no authentication and authorization by default and can be disabled. Enable SMB shares and use Microsoft Active Directory (AD) for user authentication and authorization. For more information, please see documentation on Working with Microsoft Active Directory in FSx for ONTAP.

Conclusion

In this post, we covered FSx NetApp ONTAP with information that can help FSI customers accelerate the service’s approval within these five categories: achieving compliance, data protection, isolation of computing environments, automating audits with APIs, and operational access and security.

We have a migration planning and strategies guide to help on prem NetApp customer migrate to AWS. All our other FSx for ONTAP blogs are here and dive deeper into different use cases. Deloitte, NetApp, and AWS recently had a panel discussion video about FSx for ONTAP in the financial services industry.

Be sure to visit our AWS Industries blog channel and stay tuned for more financial services news and best practices.