AWS Storage Blog
Deploying Amazon FSx for NetApp ONTAP using AWS CloudFormation
In today’s fast-paced digital landscape, the efficient management and accessibility of data stand as pivotal challenge for businesses. For a global team to efficiently collaborate on a project, they require a shared file storage that supports multiple protocols, robust data management features, and dependable data integrity. When deploying infrastructure repeatedly or in different environments, it is important that infrastructure is consistently reproducible, reducing the risk of configuration errors.
Amazon FSx for NetApp ONTAP is ideal for those looking for a fully managed solution for multi-protocol access, as it allows you to focus on core business activities instead of managing complex infrastructure. An effective way of deploying infrastructure consistently and without errors is to use Infrastructure-as-Code (IaC) tools – and you can deploy FSx for ONTAP with several.
In this blog, we discuss deployment using AWS CloudFormation, and deploy an FSx for ONTAP file system using it. CloudFormation lets you provision related AWS and third-party resources quickly and consistently, and manage them throughout their life cycles, by treating infrastructure as code. You can incorporate the code in this blog in an existing template to create, update, and delete an entire stack as a single unit. Using Drift Detection, you can keep track of changes to resources outside CloudFormation, making sure you always have the most up-to-date picture of your infrastructure.
Infrastructure-as-Code (IaC) tools
By definition IaC is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. IaC enables you to maintain the configuration or state of the infrastructure in a known way.
It is recommended to use AWS CloudFormation when all the resources you are deploying are in AWS. If you need to orchestrate multi-cloud environments, or need to use HashiCorp Terraform for other reasons, you can refer to the blog Deploying Amazon FSx for NetApp ONTAP using HashiCorp Terraform. If you already have a file system deployed and are looking for a configuration management tool to maintain the configuration in a known state, Ansible will be able to help. You can refer to How to manage Amazon FSx for NetApp ONTAP with Ansible for a brief introduction to Ansible. If you need more flexibility and want to perform operations not supported by AWS CloudFormation, or want to developer friendly automation refer to the blog Deploying Amazon FSx for NetApp ONTAP using AWS CloudFormation.
There are wide variety of tools in the Infrastructure-as-Code (IaC), configuration management and automation market. AWS CloudFormation and HashiCorp Terraform are provisioning tools. Ansible is a configuration management tool. The tool you should use depends on the challenges you need to solve and your posture in terms of automation. For performing Day-0 activities like provisioning it is recommended to use either AWS CloudFormation or HashiCorp Terraform. If customer is totally in AWS, we recommend using AWS CloudFormation because of its integration with other AWS services. If you have a need to build and manage hybrid or multi-cloud, we recommend using HashiCorp Terraform.
For performing Day-1 activities like configuring storage, we recommend that you use Ansible. Ansible offers certified modules for managing NetApp. For complete list of NetApp modules refer the collection documentation Netapp.Ontap. If you need more flexibility than what Ansible can provide, we recommend using the REST API. In some cases, you might have to use a combination of these tools. Combinations that we recommend for managing FSx for ONTAP are provisioning and configuration management (AWS CloudFormation/Terraform + Ansible) or provisioning and Orchestration (AWS CloudFormation/Terraform + REST API). Refer to the “Additional reading” section to get more insight into the tools mentioned.
Overview of components
Amazon FSx for NetApp ONTAP is a fully managed service that provides highly reliable, scalable, high-performing, and feature-rich file storage built on NetApp’s popular ONTAP file system. FSx for ONTAP provides a seamless solution for multi-protocol access, allowing the organization to focus on their core business activities instead of managing complex infrastructure. FSx for ONTAP file systems are similar to on-premises NetApp clusters. The following list identifies the main components in FSx for ONTAP:
- File system: A file system is the primary Amazon FSx resource, analogous to an on-premises ONTAP cluster
- Storage Virtual Machine (SVM): An SVM is an isolated file server with its own administrative credentials and endpoints for administering and accessing data
- Volumes: ONTAP serves data to clients and hosts from logical containers called volumes
AWS CloudFormation is an AWS native IaC service that lets you model, provision, and manage AWS and third-party resources. It uses either JSON or YAML templates. CloudFormation has many advantages including; CloudFormation is a fully managed AWS service, and state management is a built in feature. It is native to AWS, so support for new services is available faster. It’s easy to use, has custom resource support, and advanced features like drift detection and rollback capabilities are available.
It is important to note that CloudFormation is proprietary to AWS. So it only supports provisioning on AWS. CloudFormation templates also has a size limit of 51 MB or 500 resources per template. This limit can be overcome with the help of nested stacks.
The following table provides an overview of the CloudFormation feature set:
Feature | AWS CloudFormation |
Support | Native managed service |
Scope | AWS Services* |
Type of tool | Orchestration tool |
State management | Yes (manages state in stacks) |
State file format | JSON or YAML |
License and support | Free and covered under AWS support |
Change verification | Yes, using Change sets |
API Model | Native FSx API |
Language | Declarative language |
Infrastructure | Immutable* |
Rollback | Yes |
External wait conditions | Yes |
Deploying FSx for ONTAP file system using AWS CloudFormation
AWS CloudFormation enables you to provision and manage AWS and third-party resources by treating infrastructure-as-code. It gives developers and businesses an easy way to create a collection of related resources, and provision and manage them in an orderly and predictable fashion. With a few clicks, you can use a CloudFormation template to pre-configure and deploy FSx for ONTAP resources like file systems, storage virtual machines, and volumes in a standardized and repeatable way across multiple regions and accounts. Customers benefit from this by being able to architect their deployments in a variety of ways. They have more control over provisioning and extending the architecture of the FSx for ONTAP with IaC with the same configuration in multiple environments.
Prerequisites
- You will need an AWS account and associated credentials that allow you to create resources
- Amazon Virtual Private Cloud (Amazon VPC) and two associated subnets we utilize for creating the file system. We are discussing Multi-AZ file system in rest of the blog which is deployed across multiple Availability Zones (AZ)
Procedure
- AWS CloudFormation resource type
AWS::FSx::FileSystem
creates a file system in FSx for ONTAP. It takes input from the user on parameters likeStorageCapacity
,SubnetIds
, etc. The deployed file system’s configuration will depend on the parameter values provided. The listed template will create a multi-AZ file system namedOntapMultiAzFileSystemWithAllConfigs
. - AWS CloudFormation resource type
AWS::FSx::StorageVirtualMachine
creates a storage virtual machine (SVM) for an Amazon FSx for ONTAP file system. The listed template creates a Storage Virtual Machine (SVM) namedOntapStorageVirtualMachineWithAllConfigs
. An SVM is a virtual, isolated file server with its own administrative credentials and IP address for accessing data. In this example, we are using Active Directory to manage access to the file shares. Note that Active Directory is not required, as FSx for ONTAP supports Workgroup mode as well. It takes user credentials from AWS Secrets Manager.OntapStorageVirtualMachineWithAllConfigs
refers to the previously created file systemOntapMultiAzFileSystemWithAllConfigs
as one of the attributes. AWS::FSx::Volume
creates a FSx for ONTAP storage volume. The template provided creates a volume calledOntapVolumeWithAllConfigs
. It refers to the previously created resourceOntapStorageVirtualMachineWithAllConfigs
as one of the attributes. The following parameterized template furnishes all the components needed to establish an FSx for ONTAP file system, including an SVM and a volume:
Description: Creates your Amazon FSx for NetApp ONTAP.
Metadata:
Authors:
Description: Amazon Web Services
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Amazon FSx for NetApp ONTAP file system parameters
Parameters:
- FSxNName
- StorageCapacity
- SubnetIds
- SecurityGroupIds
- Iops
- AutomaticBackupRetentionDays
- PreferredSubnetId
- RouteTableIds
- EndpointIpAddressRange
- ThroughputCapacity
- FsxAdminPassword
- KmsKeyId
- Label:
default: Storage Virtual Machine for Amazon FSx for NetApp ONTAP file system parameters
Parameters:
- SVMname
- NetBiosName
- DnsIps
- DomainName
- OrganizationalUnitDistinguishedName
- ADsecretname
- RootVolumeSecurityStyle
ParameterLabels:
FSxNName:
default: Name of the Filesystem
StorageCapacity:
default: Storage Capacity
SubnetIds:
default: Subnet Ids
SecurityGroupIds:
default: Security group ID
Iops:
default: Iops
AutomaticBackupRetentionDays:
default: AutomaticBackupRetentionDays
PreferredSubnetId:
default: Preferred Subnet Id for FSxN
RouteTableIds:
default: Preferred route table for FSxN
EndpointIpAddressRange:
default: floating ip EndpointIpAddressRange
ThroughputCapacity:
default: Preferred ThroughputCapacity for FSxN
FsxAdminPassword:
default: Default password for FSxN.
KmsKeyId:
default: AWS KMS key ID for FSxN Encyption at Data at Rest
SVMname:
default: Name of the SVM for FSxN.
NetBiosName:
default: Preferred NetBiosName for FSxN
DnsIps:
default: DNS ips for the FSxN
DomainName:
default: Domain Name of the ActiveDirectoryConfiguration
OrganizationalUnitDistinguishedName:
default: Organization unit for FSxN
ADsecretname:
default: AWS Secrets Manager Secret Name
RootVolumeSecurityStyle:
default: Security Style of the Root volume.
Parameters:
FSxNName:
Description: Name of the Filesystem
Type: String
StorageCapacity:
Description: StorageCapacity for FSxN FileSystem
Default: 1024
Type: Number
SubnetIds:
Description: Subnet ids for FSxN FileSystem in comma seperated values.
Type: CommaDelimitedList
SecurityGroupIds:
Description: Security group ID for the FSxN FileSystem.
Type: CommaDelimitedList
Iops:
Description: Choose Iops of your Amazon FSx for NetApp ONTAP file system.
Default: 3072
Type: Number
AutomaticBackupRetentionDays:
Type: Number
Description: Backup of your Amazon FSx for NetApp ONTAP file system's Volumes.
PreferredSubnetId:
Description: Provide a subnet that will be preferred node of Amazon FSx for NetApp ONTAP file system's Volumes.
Type: String
RouteTableIds:
Description: Route table for the FSxN
Type: CommaDelimitedList
EndpointIpAddressRange:
Description: Private floating endpoint EndpointIpAddressRange for FSxN.
Type: String
ThroughputCapacity:
Description: Throughput for FSxN
AllowedValues: [128, 256, 512, 1024, 2048]
Type: Number
FsxAdminPassword:
Description: AWS Secrets Manager Secret Name containing Default password to login to FSxN using fsxadmin username.
Type: String
KmsKeyId:
Description: AWS KMS key ID for FSxN Encyption at Data at Rest, if you dont have Customer Managed Key Leave it black. As AWS Fsx Service create a Fsx Key by default for Encyption at rest.
Type: String
Default: ""
SVMname:
Description: Name of the SVM for FSxN
Type: String
NetBiosName:
Description: Preferred NetBiosName for FSxN,
Type: String
DnsIps:
Description: DNS ips for the FSxN. If Multiple provide comma seperated.
Type: CommaDelimitedList
DomainName:
Description: Domain Name of the ActiveDirectoryConfiguration
Type: String
OrganizationalUnitDistinguishedName:
Description: Organization unit for FSxN.
Type: String
ADsecretname:
Description: 'AWS Secrets Manager Secret Name containing password for the AD user". Validate secret is stored in format {"username" : "<your username>", "password" : "<your password>"}'
Type: String
RootVolumeSecurityStyle:
Description: Security Style of the Root volume. Either in [MIXED, UNIX, NTFS].
Type: String
AllowedValues: [MIXED, UNIX, NTFS]
Conditions:
IsKMSKeyprovided: !Not [!Equals [ !Ref KmsKeyId, "" ]]
Resources:
OntapMultiAzFileSystemWithAllConfigs:
Type: "AWS::FSx::FileSystem"
Properties:
FileSystemType: "ONTAP"
KmsKeyId: !If [ IsKMSKeyprovided, !Ref KmsKeyId, !Ref "AWS::NoValue"]
StorageCapacity: !Ref StorageCapacity
StorageType: SSD
SubnetIds: !Ref SubnetIds
SecurityGroupIds: !Ref SecurityGroupIds
OntapConfiguration:
AutomaticBackupRetentionDays: !Ref AutomaticBackupRetentionDays
DailyAutomaticBackupStartTime: "07:00"
DeploymentType: "MULTI_AZ_1"
DiskIopsConfiguration:
Iops: !Ref Iops
Mode: "USER_PROVISIONED"
PreferredSubnetId: !Ref PreferredSubnetId
RouteTableIds: !Ref RouteTableIds
EndpointIpAddressRange: !Ref EndpointIpAddressRange
ThroughputCapacity: !Ref ThroughputCapacity
WeeklyMaintenanceStartTime: "4:16:30"
FsxAdminPassword: !Join
- ':'
- - '{{resolve:secretsmanager'
- !Ref FsxAdminPassword
- 'SecretString:password}}'
Tags:
- Key: "Name"
Value: !Ref FSxNName
OntapStorageVirtualMachineWithAllConfigs:
Type: "AWS::FSx::StorageVirtualMachine"
Properties:
ActiveDirectoryConfiguration:
NetBiosName: !Ref NetBiosName
SelfManagedActiveDirectoryConfiguration:
DnsIps: !Ref DnsIps
DomainName: !Ref DomainName
OrganizationalUnitDistinguishedName: !Ref OrganizationalUnitDistinguishedName
Password: !Join
- ':'
- - '{{resolve:secretsmanager'
- !Ref ADsecretname
- 'SecretString:password}}'
UserName: !Join
- ':'
- - '{{resolve:secretsmanager'
- !Ref ADsecretname
- 'SecretString:username}}'
FileSystemId: !Ref OntapMultiAzFileSystemWithAllConfigs
Name: !Ref SVMname
RootVolumeSecurityStyle: !Ref RootVolumeSecurityStyle
Tags:
- Key: "Name"
Value: "OntapSvm"
OntapVolumeWithAllConfigs:
Type: "AWS::FSx::Volume"
Properties:
Name: "volume1"
OntapConfiguration:
JunctionPath: "/volume1"
SecurityStyle: "NTFS"
SizeInMegabytes: 10240
StorageEfficiencyEnabled: True
StorageVirtualMachineId: !Ref OntapStorageVirtualMachineWithAllConfigs
TieringPolicy:
CoolingPeriod: 31
Name: "AUTO"
Tags:
- Key: "Name"
Value: "OntapVolume"
VolumeType: "ONTAP"
You can extend the functionality of the template beyond what is natively supported by AWS CloudFormation using custom resource and resource types. For example, if you want to create a CIFS share, which is not supported natively by AWS CloudFormation, you can do that using custom resources. For more information on it you can refer to the documentation on managing resources using AWS CloudFormation resource types.
As new features and services are released rapidly, sometimes they are released without CloudFormation support. Refer to the documentation on CloudFormation Coverage Roadmap which is used as a primary source of input to our development process.
As mentioned, the CloudFormation template provided in this post creates an FSx for ONTAP file system, an SVM, and a volume. You can mount it on any machine using NFS protocol. To make it accessible over SMB/CIFS, a CIFS share needs to be created.
Cleaning up
To delete the resources created in this post, go to the AWS CloudFormation console in the management account. Choose the stack you created, and then choose Delete. For more information, see the documentation on deleting a stack on the AWS CloudFormation console.
Conclusion
In this blog, we provided an overview of AWS CloudFormation and how to create a FSx for ONTAP File system, SVM, and a volume using it. Automation not only reduces the potential for manual errors, but also saves valuable time for IT teams, enabling them to focus on more strategic tasks than managing infrastructure. In exceptional scenarios where AWS CloudFormation cannot be used, FSx for ONTAP supports other tools. More information is available in the following “Additional reading”.
If you have any comments or questions, don’t hesitate to leave them in the comments section.