How to use NetApp ONTAP REST APIs with Amazon FSx for NetApp ONTAP

Developers have been using HTTP methods such as GET, POST, and DELETE for a significant period of time to interact with RESTful APIs. The widespread adoption of these methods is largely due to their user-friendliness. Users who are transitioning from on-premises NetApp storage to Amazon FSx for NetApp ONTAP can streamline their operations using its REST API endpoints.

FSx for ONTAP is ideal for individuals seeking a fully managed file system that supports multiple protocols. FSx for ONTAP is built on NetApp’s ONTAP file system. By leveraging this service, you can prioritize your core business functions rather than dealing with intricate infrastructure management. FSx for ONTAP support for REST API enables you to incorporate the standard REST API automation across the enterprise.

In this post, we delve into the process of creating volumes and shares through the ONTAP REST API and explore how to protect them using SnapMirror. Note that while this post emphasizes APIs, managing FSx for ONTAP is also possible through the AWS Management Console and AWS Command Line Interface (AWS CLI). Additionally, NetApp management tools such as NetApp BlueXP and NetApp ONTAP CLI can be used for FSx for ONTAP management.

ONTAP REST API for automation

FSx for ONTAP file systems can be managed through REST APIs, using either the native Amazon FSx API or the ONTAP REST API. The NetApp BlueXP API, similar to the BlueXP UI, serves as a unified control plane that provides a hybrid multicloud experience for NetApp storage and data services across on-premises and cloud environments. FSx for ONTAP and its resources can be controlled to some extent using native AWS API (Amazon FSx API). On the other hand, ONTAP REST API offers comprehensive storage automation capabilities. To simplify ONTAP storage automation further, NetApp offers a Python client library for programmatic REST API consumption. Alternatively, you can create your own client library by accessing the YAML file at https://<your-cluster-mgmt-IP>/docs/api/swagger.yaml. Each of these options has its unique capabilities, but our focus is on the ONTAP REST API, which is native to the ONTAP operating system. You can access the ONTAP REST API directly using any programming language that supports REST clients, with popular choices such as Python, PowerShell, and Java. In this post, we use Python as our language of choice. Note that ONTAP REST API is different from ONTAP API (also called ZAPI), which is NetApp’s proprietary API. It is recommended not to use ZAPI, as it is not supported with future ONTAP releases.

ONTAP REST APIs do not need licensing or installation, as they are readily available within ONTAP. You can access the REST API GUI in your cluster by visiting NetApp support. This GUI provides REST endpoints that can automate your day-to-day ONTAP operations management and can be integrated with third-party tools that support REST APIs. If a direct API endpoint is unavailable, then you can use the private CLI pass-through option. For more information, refer to the ONTAP REST API documentation.

It is recommended to work with REST API if you want flexibility or a developer friendly automation mechanism. If you are new to FSx for ONTAP and need to deploy a filesystem, you can refer to the blog Deploying Amazon FSx for NetApp ONTAP using AWS CloudFormation. Alternatively, if you need orchestration for multi-cloud environments or wish to create a new FSx for ONTAP filesystem, then the blog Deploying Amazon FSx for NetApp ONTAP using HashiCorp Terraform can assist. For those with an existing filesystem and a need for configuration management, Ansible can be a valuable tool. You can find a brief introduction to Ansible in the blog How to manage Amazon FSx for NetApp ONTAP with Ansible.

There are wide variety of tools in the Infrastructure-as-Code (IaC), configuration management, and automation market. AWS CloudFormation and HashiCorp Terraform are provisioning tools. Ansible is a configuration management tool. The tool you should use depends on the challenges you need to solve and your posture in terms of automation. For performing Day-0 activities such as provisioning, it is recommended to use either CloudFormation or HashiCorp Terraform. If the user is totally in AWS, then we recommend using CloudFormation because of its integration with other AWS services. If you need to build and manage hybrid or multi-cloud, then we recommend using HashiCorp Terraform.

For performing Day-1 activities such as configuring storage, we recommend that you use Ansible. Ansible offers certified modules for managing NetApp. For a complete list of NetApp modules, refer the collection documentation Netapp.Ontap. If you need more flexibility than what Ansible can provide, then we recommend using the REST API. In some cases, you might have to use a combination of these tools. Combinations that we recommend for managing FSx for ONTAP are provisioning and configuration management (CloudFormation/Terraform + Ansible) or provisioning and Orchestration (CloudFormation/Terraform + REST API). Refer to the Additional reading section to get more insight into the tools mentioned.

Overview of components

FSx for ONTAP is a fully managed service that provides highly reliable, scalable, high-performing, and feature-rich file storage built on NetApp’s popular ONTAP file system. FSx for ONTAP provides a seamless solution for multi-protocol access, allowing the organization to focus on their core business activities instead of managing complex infrastructure. FSx for ONTAP file systems are similar to on-premises NetApp clusters. The following list identifies the main components in FSx for ONTAP:

• File system: A file system is the primary Amazon FSx resource, analogous to an on-premises ONTAP cluster
• Storage Virtual Machine (SVM): An SVM is an isolated file server with its own administrative credentials and endpoints for administering and accessing data
• Volumes: ONTAP serves data to clients and hosts from logical containers called volumes

The ONTAP REST APIs offer several significant advantages for automating and managing NetApp’s ONTAP storage systems. They seamlessly integrate with existing REST API automation frameworks. This makes it convenient to incorporate ONTAP functionality into broader automation workflows. Furthermore, ONTAP REST APIs are highly efficient, providing rapid responses to API calls while optimizing ONTAP resource usage. This efficiency is particularly valuable for tasks related to CIFS/NFS file share management and operations.

Although RESTful APIs offer several benefits in terms of compatibility and simplicity, they do come with certain considerations. For those embarking on RESTful API development from scratch, there can be a steep learning curve, especially in grasping REST principles, endpoints, and data structures.

How to automate using ONTAP REST APIs

Using REST APIs, a developer/application team can provision volumes, SMB shares, and replication rather than asking the storage team to provision when needed.

In this post we assume that you have created an FSx for ONTAP file system and SVM in different AWS Regions and established a peering relationship between a source and destination ONTAP cluster and their respective SVMs using their intercluster logical interfaces (LIFs). If any of the prerequisites are not met, then refer to the Additional reading section to configure them. In this article, we use ONTAP REST APIs to create a volume, SMB share, and enable volume-based replication with SnapMirror.

Prerequisites

The following prerequisites are necessary to continue with this post:

1. Curl running on the machine

2. Two FSx for ONTAP filesystems deployed in different AWS Regions in your AWS account

3. Have fsxadmin credentials

4. Assuming Cluster peering and SVM peering configured on both file systems

Solution walkthrough

1. Create a volume

2. Create an SMB share

3. Create the target volume

4. Create a SnapMirror relationship

5. Use the UUID from the SnapMirror create API to initialize the SnapMirror

Procedure:

To easily understand the values to plug into the Payload, we have the following aliases created. We are using the values directly in the API calls mentioned in rest of the post:

ONTAP REST APIs use a Base64 authorization token format when using Python-based programmatic requests. The following is an example of conversion process.

BASE64STRING = base64.encodebytes ((‘username’:’password’).encode()).decode().replace('\n', '')
headers = {
    'authorization': "Basic %s" % BASE64STRING,    
    'content-type': "application/json",    
    'accept': "application/json"
    }

When using the curl command, you can include the username and password directly in the command by using the -u option followed by the username and password. Replace username and password with fsxadmin credentials.

curl -u username:password [additional options and URL]

1. Create a volume

To create a volume, use the following payload format and POST API request. The payload should contain details of the SVM, volume name, volume size(bytes), and volume security style (ntfs/unix). Provide management IP of the FSxN to communicate to an API endpoint.

        payload = {
            "aggregates": [{"name": "aggr1"}],
            "svm": {"name": “SourceSVM”},
            "name": “Source_vol”,
            "size": “10737418240”, 
            "nas": {"security_style": "ntfs", "path": “/smbfileshare”}
        }

        url2 = "https://{}/api/storage/volumes".format(["ManagementIP_of_source_FSxN"])

Use the preceding input information and execute a curl command to initiate the creation of a volume as shown in the following. This execution results in a successful 202 response with the volume UUID. UUID is a unique identifier of the volume used for any subsequent calls on the volume.

curl -k -X POST -u username:password "https://<ManagementIP_of_source_FSxN>/api/storage/volumes" -H "accept: application/json" -H "Content-Type: text/plain" -d "{\"aggregates\": [{\"name\": \"aggr1\"}], \"svm\": {\"name\": \"SourceSVM\"}, \"name\": \"Source_vol\", \"size\": 10737418240, \"nas\": {\"security_style\": \"ntfs\", \"path\": \"/smbfileshare\"}}"

2. Create an SMB share

To create an SMB share, use the following payload format and POST API request. The payload should contain the SVM name, SMB Share name, and junction path of the volume.

payload = {
            "path": “/smbfileshare",
            "svm": {
                "name": “SourceSVM”,
            },
            "name": “smbfileshare”,
        }

        url3 = "https://{}/api/protocols/cifs/shares".format("ManagementIP_of_source_FSxN")

Use the preceding input details and execute a curl command, as shown in the following, to create an SMB file share. This execution results in an empty response with a 202 status code.

curl -k -X POST -u username:password "https://<ManagementIP_of_source_FSxN>/api/protocols/cifs/shares?return_timeout=0&return_records=false" -H "Content-Type: text/plain" -d "{\"path\": \"/smbfileshare\",\"svm\": {\"name\": \"SourceSVM\"},\"name\": \"smbfileshare\"}”

3. Create the target volume

Create the target volume with DP type on the target SVM by providing the volume and size of the intended target volume. Note that the destination volume must be the same size or larger than the source volume.

payload = {
    "svm.name" : “DestinationSVM”,    
    "aggregates.name" : ["aggr1"],    
    "name" : “Destination_vol”,    
    "type" : “DP”,    
    "size" : “10737418240”  
}
url = "https://{}/api/storage/volumes".format("ManagementIP_target_FSxN")

Use the preceding inputs and run a curl command using the following example to create a new DP volume on the target Filesystem

curl -k -X POST -u username:password "https://<ManagementIP_of_target_FSxN>/api/storage/volumes" -H "accept: application/json" -H "Content-Type: text/plain" -d "{\"svm.name\" : \"DestinationSVM\", \"aggregates.name\" : [\"aggr1\"], \"name\" : \"Destination_vol\", \"type\" : \"DP\", \"size\" : \"10737418240\" }"

4. Create a SnapMirror relationship

Create a SnapMirror relationship using the API and payload mentioned in the following. The response from the POST API provides you with a SnapMirror UUID.

payload = {
  'source': {
    'path': 'SourceSVM:Source_vol'}, 
  'destination': {
    'path': 'DestinationSVM:Destination_vol’
  }
}

url = "https://{}/api/snapmirror/relationships/".format("ManagementIP_of_target_FSxN")

Use the preceding inputs for the following curl command and the create SnapMirror relationship on the target file system.

curl -k -X POST -u username:password "https://<ManagementIP_of_target_FSxN>/api/snapmirror/relationships/" -H "accept: application/json"  -H "Content-Type: text/plain" -d "{\"source\" : {\"path\": \"SourceSVM:Source_vol\"}, \"destination\": {\"path\": \"DestinationSVM:Destination_vol\"}}"

This provides the response with an UUID for the relationship of the replication that is used to initiate in next step.

{
  "job": {
    "uuid": "943ea87d-63c2-11ee-b893-c1bf42ad9858",
    "_links": {
      "self": {
        "href": "/api/cluster/jobs/943ea87d-63c2-11ee-b893-c1bf42ad9858"
      }
    }
  }
}

5. Use the UUID from the SnapMirror create API to initialize the SnapMirror

Use the UUID from the SnapMirror create API to initialize the SnapMirror. Provide the UUID in end point URL so ONTAP initiates the SnapMirror replication process.

url = "https://{}/api/snapmirror/relationships/{}/transfers".format("ManagementIP_target_FSxN”, snapmirror_uuid)

Replace the URL with the target FSx for ONTAP Management IP and UUID of the SnapMirror replication created in earlier step.

curl -k -X POST -u username:password "https://<ManagementIP_of_target_FSxN>/api/snapmirror/relationships/<snapmirror_uuid>/transfers" -H "accept: application/json" -H "Content-Type: text/plain"

Cleaning up

In order to keep charges to minimum, clean up the resources in the following sequence:

• Quiesce and Stop the SnapMirror relationship.
• Delete the SMB share in the source and target file systems.
• Delete the source and target volumes.
• Delete the SVM peer relationship.
• Delete the cluster peer relationship.
• Delete both destination and source clusters if they are created as part of this post.

Conclusion

In this post, we provided an overview of ONTAP REST API with an example of consuming them for FSx for ONTAP automation. You can use Amazon FSx for NetApp ONTAP’s support of REST API to provide complete stack automation. Automation not only reduces the risk of manual errors but also enables IT teams to focus on more strategic tasks than managing infrastructure. In addition to REST API, FSx for ONTAP supports other tools for automation. More information is in the “Additional reading” section of this post.

If you have any comments or questions, don’t hesitate to leave them in the comments section.

Additional reading

Deploying Amazon FSx for NetApp ONTAP using AWS CloudFormation

Deploying Amazon FSx for NetApp ONTAP using HashiCorp Terraform

How to manage Amazon FSx for NetApp ONTAP with Ansible