AWS for SAP

AWS Transfer for SFTP for SAP file transfer workloads – part 1

This post is by Kenney Antoney Rajan, an AWS Partner Solutions Architect.

Many organizations that use enterprise resource planning (ERP) software like SAP run and maintain Secure File Transfer Protocol (SFTP) servers to securely transfer business-critical data from SAP to external partner systems. In this series of blog posts, we’ll provide steps for you to integrate your SAP Process Integration and Process Orchestration (SAP PI/PO) and SAP Cloud Platform Integration with AWS Transfer for SFTP (AWS SFTP). We’ll also show you how to use the data that AWS SFTP stores in Amazon Simple Storage Service (Amazon S3) for post-processing analytics.

Use cases

There are many SAP scenarios where an SFTP server is useful for SAP file workloads. For example:

  • Transportation industry. A company can use an SFTP server as a middle layer to place files that contain sales order data. The external transportation company processes the order information from the SFTP server to schedule transportation.
  • Retail industry. A company can send their material data from SAP to the SFTP destination for a data lake solution to process the data. The data lake solution polls and processes raw data files sent from SAP and internal sales applications, to get insights such as fast selling items by material types.

Benefits of using AWS SFTP

Regardless of industry, laws and legislation in many countries mandate that every company keep private information secure. Organizations that require an SFTP server for their SAP integration can now use AWS SFTP to distribute data between SAP ERP and external partner systems, while storing the data in Amazon S3.

AWS SFTP manages the infrastructure behind your SFTP endpoint for you. This includes automated scaling and high availability for your SFTP needs, to process business-critical information between SAP and external partner systems. To learn how to create an AWS SFTP server, see Create an SFTP Server in the AWS documentation.

Architecture overview

You can integrate your SAP environment with AWS SFTP using SAP PI/PO, which acts as an integration broker to facilitate connection between systems. The following diagram shows the high-level architecture of how your SAP PI/PO systems can integrate with AWS SFTP and perform post-processing functions.

Authentication options

To establish a connection with AWS SFTP, you’ll use SAP PI/PO authentication options:

  • SAP key-based authentication. Convert the Secure Shell (SSH) private key that’s generated as a part of the AWS SFTP server creation process to Public Key Cryptography Standards (PKCS)12 type keystore. You do this to integrate SAP PI/PO communication channels with AWS SFTP.
  • SAP PI/PO password-based authentication. Use AWS Secrets Manager to enable username- and password-based authentication. You do this to integrate SAP PI/PO communication channels with AWS SFTP.

SAP key-based authentication

You can use Open SSL to create X.509 and P12 certificates on your local SSH key pair directory, as shown in the following diagram. Enter the password and note it down for SAP keystore setup. The generated key will be in binary form.

SAP NetWeaver Administrator keystore configuration

  1. Log in to SAP NetWeaver Administrator Key Storage Views, and enter a name and description to create a new key storage view.

  1. Select Import Entry, and then choose PKCS#12 Key Pair type from the drop-down menu, to import the .p12 file created as part of the earlier Open SSL step.

 

  1. To decrypt the file and complete the import, use the same password that you used earlier, and then choose Import.

  1. Make a note of the fingerprints to integrate the SAP PI/PO systems with the AWS SFTP server to finish configuring the SAP PI/PO integration directory.

Integrating the SAP PI/PO SFTP communication channel with the AWS SFTP endpoint

Next, you’ll configure a key-based authentication method in SAP PI/PO to transfer your file workloads from SAP ERP Central Component (SAP ECC) to the AWS SFTP destination.

To test the SAP PI/PO integration, you can transfer a MATMAS material intermediate document (IDoc) from the SAP system to the AWS SFTP destination.

In this blog post, it’s assumed that you’ll configure the software and business component in the SAP PI/PO System Landscape directory, import the MATMAS IDoc structure, and map the raw IDoc structure (XML) to comma-separated value (CSV) formatted type using message, service, and operational mappings in the SAP PI/PO Enterprise Services Repository function. You can also use the raw MATMAS intermediate document structure (XML) for testing.

In addition, you’ll need to configure sender and receiver communication channels and integration configuration in the SAP PI/PO integration directory function.

In the SAP PI/PO integration directory configuration, select SFTP adapter type and update the AWS SFTP endpoint and fingerprint created during the SAP NetWeaver Administrator keystore configuration. Update the values for the authentication method and file parameter key in the SAP PI/PO communication channel screen as follows:

  • Authentication method: Private Key
  • Username: The username for the SFTP server created as part of the AWS SFTP setup process.
  • Private Key View: The key view created previously in the SAP NetWeaver Administrator keystore configuration.
  • Private Key Entry: The key entry type created previously in SAP NetWeaver Administrator keystore configuration.
  • Filename: The filename or naming convention that will be transferred from SAP to the AWS SFTP server.
  • Filepath: The Amazon S3 bucket path that’s created as part of the AWS SFTP setup process. This filepath is the AWS SFTP S3 destination where your transferred files will be stored.

SAP PI/PO password-based authentication

  1. See the Enable password authentication for AWS Transfer for SFTP using AWS Secrets Manager blog post to enable password authentication for the AWS SFTP server using AWS Secrets Manager. Note down the username and password from AWS Secrets Manager to enter in the authentication configuration of the SAP PI/PO integration directory.

  1. Update the SAP PI/PO integration directory configuration with the new AWS SFTP endpoint and fingerprint created as part of password authentication. Update the values for your authentication method and file parameter key as follows:
  • Authentication method: Password.
  • Username: The username created as part of password authentication, as mentioned in the previous step.
  • Password: The password created as part of password authentication, as mentioned in the previous step.
  • Filename: The filename or naming convention that will be transferred from SAP to the AWS SFTP server.
  • Filepath: The Amazon S3 bucket path created as part password authentication. This filepath is the SFTP destination where your transferred files will be stored.

  1. To test the integration, trigger a MATMAS IDoc using an SAP ECC BD10 transaction to send a material master XML file to the AWS SFTP S3 directory through the SAP PO/PI integration.

The file is now successfully placed in the AWS SFTP S3 directory file-path configured in the SAP PI/PO communication channel.

Post-processing analytics using AWS serverless options

AWS serverless options include the following:

  • Building serverless analytics with Amazon S3 data
  • Creating a table and exploring data

Building serverless analytics with Amazon S3 data

With your data stored in Amazon S3, you can use AWS serverless services for post-processing needs like analytics, machine learning, and archiving. Also, by storing your file content in Amazon S3, you can configure AWS serverless architecture to perform post-processing analytics without having to manage and operate servers or runtimes, either in the cloud or on premises.

To build a report on SAP material data, you can use AWS Glue, Amazon Athena, and Amazon QuickSight. AWS Glue is a fully managed data catalog and extract, transform, and load (ETL) service. As you get your AWS Glue Data Catalog data partitioned and compressed for optimal performance, you can use Amazon Athena ad-hoc query analysis on the data that’s processed by AWS Glue. You can then visualize the data using Amazon QuickSight, a fully managed visualization tool, to present the material data using pie charts.

See the Build a Data Lake Foundation with AWS Glue and Amazon S3 blog post to learn how to do the following:

  • Create data catalogs using AWS Glue
  • Execute ad-hoc query analysis on AWS Glue Data Catalog using Amazon Athena
  • Create visualizations using Amazon QuickSight

Creating a table and exploring data

Create a table with your material file stored in Amazon S3 using AWS Glue crawler. AWS Glue crawler scans through the raw data available in an S3 bucket and creates a data table with a data catalog. Using AWS Glue ETL jobs, you can transform the SAP MATMAS CSV file into parquet format, which is well suited for you to query the data with Amazon Athena.

The following figure shows how the material table named parquetsappparquet was created from the SAP MATMAS file stored in Amazon S3. For detailed steps on creating a job in parquet format, see the Build a Data Lake Foundation with AWS Glue and Amazon S3 blog post.

After completing the data transformation using AWS Glue, select the Amazon Athena service from the AWS Management Console and use Athena Query Editor to execute a SQL query on the SAP material table created in the earlier step.

Amazon QuickSight is a data visualization service that you can use to analyze data. Create a new Amazon Athena data source using Amazon QuickSight, and build a visualization of your material data. In the following example, you can see the count of materials by material type using Amazon QuickSight visualization. For more detailed instructions, see the Amazon QuickSight User Guide.

Conclusion

In part 1 of this blog series, we’ve shown how to integrate SAP PI/PO systems with AWS SFTP and how to use AWS analytics solutions for post-processing analytics. We’ve used key AWS services such as AWS SFTP, AWS Secrets Manager, AWS Glue, Amazon Athena, and Amazon QuickSight. In part 2, we’ll talk about SAP Cloud Platform integration with AWS SFTP for your file-based workloads.