Overview
One of the main challenges in building generative AI applications is complex cloud setup and the need for deep AI expertise. Generative AI Application Builder on AWS simplifies this process, helping you develop, test, and deploy AI applications without extensive AI knowledge. This solution speeds up your AI development by easily incorporating your business data, comparing the performance of large language models (LLMs), running multi-step tasks through AI Agents, quickly building extensible applications, and deploying them with enterprise-grade architecture. Generative AI Application Builder comes with a ready-to-use generative AI chatbot and API that can be quickly integrated into your business processes or applications.
This solution includes integrations with Amazon Bedrock and its LLMs in addition to LLMs deployed on Amazon SageMaker. It uses Amazon Bedrock tools for Retrieval Augmented Generation (RAG) to enhance AI responses, Amazon Bedrock Guardrails to implement safeguards and reduce hallucinations, and Amazon Bedrock Agents to create workflows for complex tasks. You can also connect to other AI models using LangChain or AWS Lambda. Start with the simple, no-code wizard to build AI applications for conversational search, AI-generated chatbots, text generation, and text summarization.
Benefits
This solution allows users to experiment quickly by removing the heavy lifting required to deploy multiple instances with different configurations and compare outputs and performance. Experiment with multiple configurations of various LLMs, prompt engineering, enterprise knowledge bases, guardrails, AI agents, and other parameters.
With pre-built connectors to a variety of LLMs, such as models available through Amazon Bedrock, this solution gives you the flexibility to deploy the model of your choice, as well as the AWS and leading FM services you prefer. You can also enable Amazon Bedrock Agents to fulfill various tasks and workflows.
Built with AWS Well-Architected design principles, this solution offers enterprise-grade security and scalability with high availability and low latency, ensuring seamless integration into your applications with high performance standards.
Extend this solution’s functionality by integrating your existing projects or natively connecting additional AWS services. Because this is an open-source application, you can use the included LangChain orchestration layer or Lambda functions to connect with the services of your choice.
Technical details
You can automatically deploy this architecture using the implementation guide and the accompanying AWS CloudFormation template that deploys three separate architectures:
- Agent Use Case – The Agent Use Case enables users to hand off tasks for completion using Amazon Bedrock Agents. You can select a model, write a few instructions in natural language, and Amazon Bedrock AI Agents will analyze, orchestrate, and complete the tasks by connecting to your data sources, or other APIs, to fulfill your request.
- Text Use Case – The Text Use Case enables users to experience a natural language interface using generative AI. This use case can be integrated into new or existing applications, and is deployable through the Deployment Dashboard or independently through a provided URL.
- Deployment Dashboard – The Deployment Dashboard is a web UI that serves as a management console for admin users to view, manage, and create their use cases. This dashboard enables customers to rapidly experiment, iterate, and deploy generative AI applications using multiple configurations of LLMs and data.
-
Agent Use Case
-
Text Use Case
-
Deployment Dashboard
-
Agent Use Case
-
Click to enlarge
Step 1
Business users deploy the use case using the Deployment dashboard. Business users sign in to the use case UI.
Step 2
Amazon CloudFront delivers the web UI, which is hosted in an Amazon Simple Storage Service (Amazon S3) bucket.
Step 3
The web UI uses a WebSocket integration built using Amazon API Gateway. The API Gateway is backed by a custom AWS Lambda Authorizer function, which returns the appropriate AWS Identity and Access Management (IAM) policy based on the Amazon Cognito group to which the authenticating user belongs.Step 4
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.Step 5
The solution passes the business user’s incoming requests from API Gateway to an Amazon Simple Queue Service (Amazon SQS) queue, and then to the Lambda function. The queue enables the asynchronous operation of the API Gateway to Lambda integration.The queue passes connection information to the Lambda function, which then posts results directly back to the API Gateway WebSocket connection to support long running inference invocations.
Step 6
The Lambda function uses Amazon DynamoDB to get the use case configurations as needed.
Step 7
The Lambda function uses the user input and relevant use case configurations to build and send a request payload to Amazon Bedrock Agents to fulfill the user intent.Step 8
When the response comes back from Amazon Bedrock Agents, the Lambda function sends the response back through the API Gateway WebSocket to be consumed by the client application.Step 9
Using Amazon CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the deployment’s performance and operational health.
Step 1
Business users deploy the use case using the Deployment dashboard. Business users sign in to the use case UI.
Step 2
Amazon CloudFront delivers the web UI, which is hosted in an Amazon Simple Storage Service (Amazon S3) bucket.
Step 3
The web UI uses a WebSocket integration built using Amazon API Gateway. The API Gateway is backed by a custom AWS Lambda Authorizer function, which returns the appropriate AWS Identity and Access Management (IAM) policy based on the Amazon Cognito group to which the authenticating user belongs.Step 4
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.Step 5
The solution passes the business user’s incoming requests from API Gateway to an Amazon Simple Queue Service (Amazon SQS) queue, and then to the Lambda function. The queue enables the asynchronous operation of the API Gateway to Lambda integration.The queue passes connection information to the Lambda function, which then posts results directly back to the API Gateway WebSocket connection to support long running inference invocations.
Step 6
The Lambda function uses Amazon DynamoDB to get the use case configurations as needed.
Step 7
The Lambda function uses the user input and relevant use case configurations to build and send a request payload to Amazon Bedrock Agents to fulfill the user intent.Step 8
When the response comes back from Amazon Bedrock Agents, the Lambda function sends the response back through the API Gateway WebSocket to be consumed by the client application.Step 9
Using Amazon CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the deployment’s performance and operational health.
-
Text Use Case
-
Click to enlarge
Step 1
Admin users deploy the use case using the Deployment Dashboard. Business users log in to the use case UI.
Step 2
CloudFront delivers the web UI which is hosted in an S3 bucket.Step 3
The web UI leverages a WebSocket integration built using API Gateway. API Gateway is backed by a custom Lambda Authorizer function, which returns the appropriate IAM policy based on the Amazon Cognito group to which the authenticating user belongs. The policy is stored in DynamoDB.Step 4
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.
Step 5
Incoming requests from the business user are passed from API Gateway to an Amazon SQS queue and then to the LangChain Orchestrator. The LangChain Orchestrator is a collection of Lambda functions and layers that provide the business logic for fulfilling requests from the business user.The queue enables the asynchronous operation of the API Gateway to Lambda integration. The queue passes connection information to the Lambda functions, which will then post results directly back to the API Gateway WebSocket connection to support long running inference calls.
Step 6
The LangChain Orchestrator uses DynamoDB to get the configured LLM options and necessary session information (such as the chat history).Step 7
If the deployment has a knowledge base configured, then the LangChain Orchestrator leverages Amazon Kendra or Amazon Bedrock Knowledge Bases to run a search query to retrieve document excerpts.Step 8
Using the chat history, query, and context from the knowledge base, the LangChain Orchestrator creates the final prompt and sends the request to the LLM hosted on Amazon Bedrock or Amazon SageMaker.Step 9
When the response comes back from the LLM, the LangChain Orchestrator streams the response back through the API Gateway WebSocket to be consumed by the client application.Step 10
Using CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the deployment’s performance and operational health.Step 1
Admin users deploy the use case using the Deployment Dashboard. Business users log in to the use case UI.
Step 2
CloudFront delivers the web UI which is hosted in an S3 bucket.Step 3
The web UI leverages a WebSocket integration built using API Gateway. API Gateway is backed by a custom Lambda Authorizer function, which returns the appropriate IAM policy based on the Amazon Cognito group to which the authenticating user belongs. The policy is stored in DynamoDB.Step 4
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.
Step 5
Incoming requests from the business user are passed from API Gateway to an Amazon SQS queue and then to the LangChain Orchestrator. The LangChain Orchestrator is a collection of Lambda functions and layers that provide the business logic for fulfilling requests from the business user.The queue enables the asynchronous operation of the API Gateway to Lambda integration. The queue passes connection information to the Lambda functions, which will then post results directly back to the API Gateway WebSocket connection to support long running inference calls.
Step 6
The LangChain Orchestrator uses DynamoDB to get the configured LLM options and necessary session information (such as the chat history).Step 7
If the deployment has a knowledge base configured, then the LangChain Orchestrator leverages Amazon Kendra or Amazon Bedrock Knowledge Bases to run a search query to retrieve document excerpts.Step 8
Using the chat history, query, and context from the knowledge base, the LangChain Orchestrator creates the final prompt and sends the request to the LLM hosted on Amazon Bedrock or Amazon SageMaker.Step 9
When the response comes back from the LLM, the LangChain Orchestrator streams the response back through the API Gateway WebSocket to be consumed by the client application.Step 10
Using CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the deployment’s performance and operational health. -
Deployment Dashboard
-
Click to enlarge
Step 1
Admin users log in to the Deployment Dashboard UI.Step 2
CloudFront delivers the web UI which is hosted in an S3 bucket.
Step 3
AWS WAF protects the APIs from attacks. This solution configures a set of rules called a web access control list (web ACL) that allows, blocks, or counts web requests based on configurable, user defined web security rules and conditions.
Step 4
The web UI leverages a set of REST APIs that are exposed using API Gateway.
Step 5
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway. A DynamoDB table stores the IAM policy of authorized users.Step 6
Lambda provides the business logic for the REST endpoints. This Backing Lambda function manages and creates the necessary resources to perform use case deployments using AWS CloudFormation.Step 7
DynamoDB stores the list of deployments.Step 8
When a new use case is created by the admin user, the Backing Lambda function initiates a CloudFormation stack creation event for the requested use case.Step 9
All of the LLM configuration options provided by the admin user in the deployment wizard are saved in DynamoDB. The deployment uses this DynamoDB table to configure the LLM at runtime.
Step 10
Using CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the solution's performance and operational health.
Step 1
Admin users log in to the Deployment Dashboard UI.Step 2
CloudFront delivers the web UI which is hosted in an S3 bucket.
Step 3
AWS WAF protects the APIs from attacks. This solution configures a set of rules called a web access control list (web ACL) that allows, blocks, or counts web requests based on configurable, user defined web security rules and conditions.
Step 4
The web UI leverages a set of REST APIs that are exposed using API Gateway.
Step 5
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway. A DynamoDB table stores the IAM policy of authorized users.Step 6
Lambda provides the business logic for the REST endpoints. This Backing Lambda function manages and creates the necessary resources to perform use case deployments using AWS CloudFormation.Step 7
DynamoDB stores the list of deployments.Step 8
When a new use case is created by the admin user, the Backing Lambda function initiates a CloudFormation stack creation event for the requested use case.Step 9
All of the LLM configuration options provided by the admin user in the deployment wizard are saved in DynamoDB. The deployment uses this DynamoDB table to configure the LLM at runtime.
Step 10
Using CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the solution's performance and operational health.
- Publish Date
Related content
The easiest way to build and scale generative AI applications with foundation models.
This Guidance demonstrates how to deploy a generative artificial intelligence (AI) model provided by Amazon SageMaker JumpStart to create an asynchronous SageMaker endpoint with the ease of the AWS Cloud Development Kit (AWS CDK).
This Guidance demonstrates how to build an application enabling users to ask questions directly of relational databases using natural language queries (NLQ).
Boost productivity, build differentiated experiences, and innovate faster with AWS.
This case study demonstrates how broadcast company Megamedia created a generative AI–powered chatbot to simplify access to important public information using AWS.