Accelerating data ingestion and on-boarding with new AWS Supply Chain capabilities

AWS Supply Chain has added several exciting enhancements, including generative AI, to simplify the data ingestion process and improve your application on-boarding and setup experience. AWS Supply Chain unifies supply chain data, provides machine learning-powered actionable insights, and offers built-in contextual collaboration to help you mitigate risks, lower costs, and increase resilience. The supply chain data lake (SCDL) is the data foundation that enables end-to-end visibility, improves demand forecasting accuracy, and increases supply chain resilience. The SCDL provides pre-built capabilities to ingest, transform, and store data from fragmented data systems into a high-quality standardized data model.

This blog post summarizes the recent releases that focus on accelerating data ingestion, streamlining customer on-boarding and configuration, and enhancing data quality capabilities. They include capabilities like automated network setup, simplified data transformations, and the ability to configure customized data extraction rules.

New releases

Generative AI-powered data association

AWS Supply Chain now uses a generative AI-powered data on-boarding agent that increases the speed and ease of on-boarding data by minimizing manual data integration. You simply extract and upload your raw data to Amazon Simple Storage Service (Amazon S3). Or, upload data from any source in its native format directly through the AWS Supply Chain user interface (UI). The on-boarding agent automatically generates transformation recipes and provides a SQL editor in the UI for building custom transformations. Built-in process checks ensure you have the data needed for your selected tasks.

The data on-boarding agent solves a key challenge: integrating data from multiple sources. It automatically transforms data from any format into the AWS Supply Chain data model. A guided, module-driven workflow informs you about required datasets based on your selected AWS Supply Chain modules (Demand Planning, Supply Planning, Insights, N-Tier Visibility, and Sustainability). This demo covers the entire process and shares additional details.

Data orchestration framework

AWS Supply Chain now supports a new data orchestration framework that enhances the on-boarding experience for SAP S/4HANA ERP customers. This framework further simplifies the data ingestion process resulting in faster setup.

Key features include:

• Simplified data transformation: You can now create custom transformations by editing SQL files, reducing the number of complex configuration recipes from 85 to 21 and shrinking their size by up to 90%.
• New data staging layer: This layer acts as a reference source by storing original and unstructured data during ingestion. It maintains dependency graphs to ensure data completeness and enables parallel ingestion from different sources. This increases data ingestion speed and reduces errors.
• Performance improvements: Horizontal cell-based scalability allows distributing workload across multiple nodes, enabling parallel processing and reducing data ingestion wait time by 80%.

Automated Private-Link setup

We’ve automated the AWS PrivateLink setup process for S/4HANA connections to accelerate on-boarding times. The automated setup using an AWS CloudFormation template reduces the end-to-end process to under 30 minutes. The CloudFormation template navigates the customer through a series of on-boarding steps and is available via the open source GitHub library.

Data quality engine process

This new capability leverages the new data orchestration framework’s data quality framework to run asynchronous data quality check each time data is ingested into the SCDL. The data quality validation ensures the accuracy, consistency, and completeness of ingested data. Data quality and validation results are accessible to users either through an event and API access, a quality report in the user’s S3 bucket, and through a new data quality user interface (UI) that will display ingestion and module errors.

Customized SAP table extracts

Instead of a one-size-fits-all data extraction catalog, customers can now define their own customized SAP tables, fields, frequencies and filters – reducing the number of required data flows from 55 down to as few as 4 for customers focused on specific application modules. This new process allows customers to add customized tables and fields as SAP ERP systems are often highly customized.

Conclusion

We innovate based on your feedback through our working backwards approach. These latest releases demonstrate our commitment to accelerating time-to-value, simplifying and accelerating data integration process, streamlining on-boarding, and enhancing data quality for our customers.

Please visit our documentation page for additional details and more detailed setup and configuration instructions. Please also visit our AWS Workshops page for a self-paced technical overview and visit AWS Supply Chain to learn more about unlocking your supply chain data at speed and scale.