New generative AI-powered data on-boarding for AWS Supply Chain

In the fast-paced world of supply chain management, where economic shifts and evolving customer preferences pose constant challenges, prioritizing efficiency and adaptability are crucial to effectively meeting end-customer demands. Organizations are increasingly turning to automated solutions to streamline supply chain management functions that reduce manual tasks, increase efficiency, and improve overall performance. However, fragmented data scattered across different platforms hinders seamless information flow, true supply chain visibility, and limits actionable insights. Additionally, integrating data from multiple sources can be a challenging and time-consuming task. Manually transforming data into the required formats often leads to inefficiencies and delays in gaining valuable insights.

AWS Supply Chain recently launched a new generative AI-powered data onboarding agent built on Amazon Bedrock to seamlessly integrate data from disparate sources. This onboarding agent increases the speed and ease of onboarding customer data by eliminating the need to manually transform data from its native format into the AWS Supply Chain Data Lake (SCDL) canonical data model format. Customers can simply extract and drop their raw data into Amazon Simple Storage Service (S3) or even upload data extracted from any source in its native format directly from the AWS Supply Chain user interface (UI). This purpose-built data management system plays a vital role by unifying and centralizing diverse data sources, enabling informed decision-making for effective supply chain management.

Key customer benefits include:

1. Streamlined integration: Generative AI automates the transformation of customer data from any native format to the AWS Supply Chain data model, eliminating tedious and manual work.
2. Guided experience: A simplified, module-driven UI streamlines data ingestion, allowing customers to select desired AWS Supply Chain modules and ingest only the required datasets. Built-in checks ensure you have the data needed for specific tasks.
3. Effortless data upload: The new “Browse and Upload” feature enables seamless data upload from any source in its native format, without navigating to the S3 console.
4. Flexible transformations: While auto-association generates transformation recipes, customers can also build custom transformations using the responsive SQL editor within the UI.

This new capability is now generally available and this blog post will walk you through the on-boarding and setup processes.

Simple four step on-boarding process

The auto-association capability associates customer data in any native format to the AWS Supply Chain data model. Large language models (LLMs) read the headers from the input files to associate them to the tables in the AWS Supply Chain canonical data model (CDM), benefiting customers who extract data in tabular formats from their databases (e.g., Forecast data, Product data, Sales Data). The onboarding workflow is meticulously designed as a user-friendly four-step process ensuring precision, simplicity, and providing comprehensive support to customers throughout the data integration journey.

These four steps are shown on the diagram below. You will first select your source system. Next, you will upload the required source files for the relevant AWS Supply Chain modules to streamline data gathering. The third step creates source flows for the uploaded tables, preparing the data for integration. Finally, the fourth step automatically associates your source tables with AWS Supply Chain destination tables using destination flows, ensuring a seamless transition.

Each step is described in detail in the next section.

Prerequisites

For this blog post, it is assumed that you understand the use of the mentioned services and you have the following prerequisites:

1. AWS account with the listed services enabled.
2. The ability to create and modify AWS Identity and Access Management (IAM) roles. This is required to create policies and permissions (particularly services related to AWS Glue).
3. Access to source systems and the ability to extract .csv files from these systems.

Step 1 – Choose your Source

You can now ingest data from any source. This means that AWS Supply Chain can now accommodate different and common data inputs from traditional, relational databases to cloud-based databases. This approach ensures we can easily onboard new data sources as they emerge.

You can upload CSV data in its native format directly within the web application. This capability extends beyond currently supported formats like SAP, EDI, and AWS Supply Chain CDM. The data source selection screen is shown below.

Step 2 – Upload your Data

The next step involves uploading source files specific to the modules you’re using. You can now choose specific AWS Supply Chain modules to see necessary datasets. This eliminates the need to ingest all datasets or consult user guides, reducing back-and-forth interactions. The module selection screen is shown below.

The enhanced user experience also enables customers to upload data directly from the UI using an embedded browse and upload feature. This approach mimics the familiar AWS S3 experience and is as depicted in the screenshot below. This integration eliminates the need for users to navigate between the web application and the S3 console.

Step 3 – Source Flows

After uploading source data, the system automatically infers the source table schema, eliminating manual effort. It then creates source flows, enabling recurring data imports against these flows without intervention. The source data are also simultaneously staged into the data lake. You don’t have to take any action during this step.

Step 4 – Generative AI-powered auto-association

In the final step, the auto-association capability associates your source tables with the AWS Supply Chain destination table. LLMs read the headers from the input files to associate source headers to the headers in AWS Supply Chain’s canonical data model (CDM). This automated data mapping streamlines operations, reduces errors, saves time, and improves overall efficiency. The auto-association process is shown in the following screenshot, where the product data is automatically associated to AWS Supply Chain data model.

In the final step, the auto-association capability associates your source tables with the AWS Supply Chain destination table. LLMs read the headers from the input files to associate source headers to the tables and fields in AWS Supply Chain’s canonical data model (CDM). Our system utilizes Chain-of-Thought (CoT) prompting to break the task into manageable components. This methodical approach enables the LLMs to comprehensively analyze the structure and content of both the source data and the AWS Supply Chain data model, ensuring precise mappings. The auto-association process is shown in the following screenshot, where the product data is automatically associated to AWS Supply Chain data model.

Built-in Responsive SQL Editor

The auto-association process generates SQL based on the mapping between customer data and AWS Supply Chain data models. This automation eliminates manual SQL maintenance and reduces errors. The mapping and SQL are responsive, ensuring edits to either component reflect in the other. A sample SQL for product mapping is shown in screenshot below.

You can review this entire process in our data association demo here.

Conclusion

AWS Supply Chain’s generative AI-powered data onboarding agent streamlines supply chain management by seamlessly integrating data from disparate sources. This advanced yet simple solution automates data transformation from native formats into the AWS Supply Chain canonical data model. By unifying and centralizing fragmented data, it enables enhanced visibility, actionable insights, and informed decision-making for effective supply chain operations.

The key benefits of this innovative solution include streamlined integration through automated data transformation, a guided module-driven UI experience, nearly effortless data upload capabilities, flexible transformation options, and end-to-end evaluation using pre-populated datasets. This four-step onboarding process leverages large language models to automate data mapping, reducing manual effort and errors while improving overall efficiency.

As organizations navigate dynamic market conditions and evolving customer demands, AWS Supply Chain’s data onboarding agent emerges as a key enabler for modern businesses. By prioritizing efficiency, adaptability, and proactive supply chain management, this solution positions companies to effectively meet end-customer demands and gain a competitive edge in today’s fast-paced business landscape.

AWS Supply Chain is available without any up-front licensing fees or long-term commitments. It provides an elastic solution that scales with your needs and is available to all AWS Supply Chain customers. Visit AWS Supply Chain to learn more and get started. You can also visit the AWS Workshop Studio for a self-paced technical overview of creating an instance, ingesting data, navigating the user interface, creating insights, and generating demand plans.