Overview
Generative AI Application Builder on AWS facilitates the development, rapid experimentation, and deployment of generative artificial intelligence (AI) applications without requiring deep experience in AI. This AWS Solution accelerates development and streamlines experimentation by helping you ingest your business-specific data and documents, evaluate and compare the performance of large language models (LLMs), rapidly build extensible applications, and deploy those applications with an enterprise-grade architecture.
Generative AI Application Builder on AWS includes integrations with Amazon Bedrock and its included LLMs, and LLMs deployed on Amazon SageMaker. Additionally, this solution has pre-built connectors to providers such as Anthropic and Hugging Face and enables connections to your choice of model using LangChain or AWS Lambda. Start with the no-code deployment wizard to build generative AI applications for conversational search, AI-generated chatbots, text generation, and text summarization.
Benefits
This solution allows users to experiment quickly by removing the heavy lifting required to deploy multiple instances with different configurations and compare outputs and performance. Experiment with multiple configurations of various LLMs, prompt engineering, enterprise knowledge bases, and other parameters.
With pre-built connectors to a variety of LLMs, such as models available through Amazon Bedrock and external providers Anthropic and Hugging Face, this solution gives you the flexibility to deploy the model of your choice, as well as the AWS and leading FM services you prefer.
Built with AWS Well-Architected design principles, this solution offers enterprise-grade security and scalability with high availability and low latency, ensuring seamless integration into your applications with high performance standards.
Extend this solution’s functionality by integrating your existing projects or natively connecting additional AWS services. Because this is an open-source application, you can use the included LangChain orchestration layer or Lambda functions to connect with the services of your choice.
Technical details
You can automatically deploy this architecture using the implementation guide and the accompanying AWS CloudFormation template that deploys two separate architectures:
- Deployment Dashboard – The Deployment Dashboard is a web user interface (UI) that serves as a management console for admin users to view, manage, and create their use cases. This dashboard enables customers to rapidly experiment, iterate, and deploy generative AI applications using multiple configurations of LLMs and data.
- Text Use Case – The Text Use Case enables users to experience a natural language interface using generative AI. This use case can be integrated into new or existing applications, and is deployable through the Deployment Dashboard or independently through a provided URL.
-
Deployment Dashboard
-
Text Use Case
-
Deployment Dashboard
-
Click to enlarge
Step 1
Admin users log in to the Deployment Dashboard user interface.
Step 2
Amazon CloudFront delivers the web UI which is hosted in an Amazon Simple Storage Service (Amazon S3) bucket.Step 3
AWS WAF protects the APIs from attacks. This solution configures a set of rules called a web access control list (web ACL) that allows, blocks, or counts web requests based on configurable, user defined web security rules and conditions.
Step 4
The web UI leverages a set of REST APIs that are exposed using Amazon API Gateway.
Step 5
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.
Step 6
AWS Lambda provides the business logic for the REST endpoints. This Backing Lambda function manages and creates the necessary resources to perform use case deployments using AWS CloudFormation.Step 7
Amazon DynamoDB acts as a configuration store for the deployment details.Step 8
When a new use case is created by the admin user, the Backing Lambda function initiates a CloudFormation stack creation event for the requested use case.Step 9
If the configured deployment uses an LLM accessed outside of AWS, then an API key is required, and a secret is created in AWS Secrets Manager to store the API key.Step 10
All of the LLM configuration options provided by the admin user in the deployment wizard are saved in Parameter Store, a capability of AWS Systems Manager. The deployment uses this Parameter Store to configure the LLM at runtime.Step 11
Using Amazon CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the solution's performance and operational health.
Step 1
Admin users log in to the Deployment Dashboard user interface.
Step 2
Amazon CloudFront delivers the web UI which is hosted in an Amazon Simple Storage Service (Amazon S3) bucket.Step 3
AWS WAF protects the APIs from attacks. This solution configures a set of rules called a web access control list (web ACL) that allows, blocks, or counts web requests based on configurable, user defined web security rules and conditions.
Step 4
The web UI leverages a set of REST APIs that are exposed using Amazon API Gateway.
Step 5
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.
Step 6
AWS Lambda provides the business logic for the REST endpoints. This Backing Lambda function manages and creates the necessary resources to perform use case deployments using AWS CloudFormation.Step 7
Amazon DynamoDB acts as a configuration store for the deployment details.Step 8
When a new use case is created by the admin user, the Backing Lambda function initiates a CloudFormation stack creation event for the requested use case.Step 9
If the configured deployment uses an LLM accessed outside of AWS, then an API key is required, and a secret is created in AWS Secrets Manager to store the API key.Step 10
All of the LLM configuration options provided by the admin user in the deployment wizard are saved in Parameter Store, a capability of AWS Systems Manager. The deployment uses this Parameter Store to configure the LLM at runtime.Step 11
Using Amazon CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the solution's performance and operational health.
-
Text Use Case
-
Click to enlarge
Step 1
Admin users deploy the use case using the Deployment Dashboard. Business users log in to the use case UI.
Step 2
CloudFront delivers the web UI which is hosted in an S3 bucket.Step 3
The web UI leverages a WebSocket integration built using API Gateway. The API Gateway is backed by a custom Lambda Authorizer function, which returns the appropriate AWS Identity and Access Management (IAM) policy based on the Amazon Cognito group the authenticating user is part of.Step 4
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.
Step 5
The LangChain Orchestrator is a collection of Lambda functions and layers that provide the business logic for fulfilling requests from the business user.
Step 6
The LangChain Orchestrator uses Parameter Store and DynamoDB to get the configured LLM options and necessary session information, such as the chat history.Step 7
If the deployment has knowledge base enabled, then the LangChain Orchestrator leverages Amazon Kendra to run a search query to retrieve document excerpts.Step 8
Using the chat history, query, and context from Amazon Kendra, the LangChain Orchestrator creates the final prompt and sends the request to the LLM hosted on Amazon Bedrock or Amazon SageMaker.Step 9
If using an LLM accessed outside of AWS, the API key is stored in Secrets Manager. This API key must be obtained before making the API call to the model provider.Step 10
When the response comes back from the LLM, the LangChain Orchestrator streams the response back through the API Gateway WebSocket to be consumed by the client application.
Step 11
Using CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the deployment’s performance and operational health.
Step 1
Admin users deploy the use case using the Deployment Dashboard. Business users log in to the use case UI.
Step 2
CloudFront delivers the web UI which is hosted in an S3 bucket.Step 3
The web UI leverages a WebSocket integration built using API Gateway. The API Gateway is backed by a custom Lambda Authorizer function, which returns the appropriate AWS Identity and Access Management (IAM) policy based on the Amazon Cognito group the authenticating user is part of.Step 4
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.
Step 5
The LangChain Orchestrator is a collection of Lambda functions and layers that provide the business logic for fulfilling requests from the business user.
Step 6
The LangChain Orchestrator uses Parameter Store and DynamoDB to get the configured LLM options and necessary session information, such as the chat history.Step 7
If the deployment has knowledge base enabled, then the LangChain Orchestrator leverages Amazon Kendra to run a search query to retrieve document excerpts.Step 8
Using the chat history, query, and context from Amazon Kendra, the LangChain Orchestrator creates the final prompt and sends the request to the LLM hosted on Amazon Bedrock or Amazon SageMaker.Step 9
If using an LLM accessed outside of AWS, the API key is stored in Secrets Manager. This API key must be obtained before making the API call to the model provider.Step 10
When the response comes back from the LLM, the LangChain Orchestrator streams the response back through the API Gateway WebSocket to be consumed by the client application.
Step 11
Using CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the deployment’s performance and operational health.
- Publish Date
Related content
The easiest way to build and scale generative AI applications with foundation models.
This Guidance demonstrates how to deploy a generative artificial intelligence (AI) model provided by Amazon SageMaker JumpStart to create an asynchronous SageMaker endpoint with the ease of the AWS Cloud Development Kit (AWS CDK).
This Guidance demonstrates how to build an application enabling users to ask questions directly of relational databases using natural language queries (NLQ).
Boost productivity, build differentiated experiences, and innovate faster with AWS.