Create a Fullstack, Sample Web App powered by Amazon Bedrock

In this post we will walk you through creating a fullstack, sample web app that uses Amazon Bedrock to create generative AI-powered application features and user experiences. Your sample app will include:

1. A React frontend web application, built with Amplify UI components, that supports both end user chat interactions with Amazon Bedrock foundational models, and configuration of generative AI agents by an administrative user.
2. A real-time, AWS AppSync WebSockets API that provides asynchronous communications between your frontend and Amazon Bedrock, which also stores and manages conversation history in Amazon DynamoDB.
3. An example of how you can connect your Amazon Bedrock foundational models to your own private data with AWS AppSync and a GraphQL API.
4. AWS Cloud Development Kit (CDK) infrastructure code to easily create and modify this sample application.

Build this sample application to learn how generative AI is transforming the way applications interface with data, and the new challenges that application developers building with generative AI services like Amazon Bedrock need to address. Then, use this sample application as the foundation for your own solution.

Video example of what we will create

Once you complete the steps contained in this post you will have a fully functional web application that you can use to create and configure generative AI agents, and chat with Amazon Bedrock foundational models (FMs). Amazon Bedrock, a fully managed service, offers a choice of high-performing FMs from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon via a single API, along with a broad set of capabilities you need to build generative AI applications. Watch the following video to see what your sample application will look like.

Generative AI-powered application architecture

The high level system diagram described below is what powers this sample app. In the following sections we’ll walk you through how you can easily set up this sample app yourself, leveraging the provided AWS Cloud Development Kit (CDK) construct.

Prerequisites

To follow this tutorial we will assume you have the following configured:

• An AWS Account with access to AWS IAM, DynamoDB, AWS Cognito, AWS AppSync, AWS Lambda, AWS CloudWatch etc and AWS Bedrock. Note that Bedrock requires submitting for approval for Anthropic models.
• Node.js v18 or higher installed
• AWS CDK 2.103 or higher installed
• CDK bootstrapped in region you wish to deploy
• Git
• Docker

Setup

Cloning the repository

We need to first get the example code pulled onto our computer, you can go here to see the code needed for this tutorial.

Download it with:

git clone https://github.com/aws-samples/appsync-bedrock-sample-with-ai-agent.git

After you clone this project, you’ll have a project with the following folders on your computer:

cdk-infrastructure
handler-claude-agent
handler-claude-simple
handler-claude-websocket
playground

Open the project root with your IDE of choice.

Deploying the infrastructure

If you have not, boot the Docker daemon. This can be done simply by either opening Docker Desktop or run Docker . More information can be found on the docker docs. Docker is used to build the python lambda functions in CDK.

To deploy the example architecture, run the following commands. This will assume your local environment has the correct AWS configurations and permissions to create resources.

# Start with infrastructure
cd cdk-infrastructure
npm install

# Build typescript resolvers into javascript
npm run build

# Deploy Infrastructure
npm run deploy

These commands will do the following in order:

• Install the dependencies for the AWS CDK package
• Compile all the .ts AppSync resolver source files into .js files
• Deploy the AWS CDK infrastructure (this make take a few minutes on the first deployment as it includes Docker container builds)
• Upload example data to the generated Amazon DynamoDB tables. Note above that the deploy script “npm run deploy” includes additional scripting to populate these DynamoDB tables that is not included in the direct command “cdk deploy”.

Requesting Model Access

AWS Bedrock allows accounts to request access to various Generative AI Models. This requires manual action on the AWS Bedrock console. This playground makes use of Claude V2 from Anthropic within the foundation model handler Lambda functions.

The AWS Bedrock Console can be accessed through your AWS Console and searching for bedrock. Verify you have selected the correct region corresponding to the region you deployed your CDK application. Default is to deploy to US-EAST-1. Note that the AWS Bedrock access is region specific. Selecting model access should allow you to see a list like below.

You can then submit a use case to get approved for Claude V2 Model Access.

Once approved, your approval should be shown. Shown below is the indication for “Access granted”.

Using the Sample App Local Playground

Once your infrastructure is deployed, navigate back to the root of the repo and run the following commands to launch the sample web app locally.

# Move to the playground
cd playground

# Load in cdk stack metadata
npm run configure

# Install dependencies with yarn
yarn

# Boot site
npm run start

This will start a local React server on localhost:3000 and you should see a page like this:

The sample app experience manages access through an Amazon Cognito User Pool, which was deployed to your AWS account as part of the AWS CDK deployment. Creating an account will add your user to the user pool in your own AWS account, automatically send you a confirmation email, and let you sign in. Your username and password are stored in the Amazon Cognito User Pool, deployed to your account as part of the AWS CDK setup, and do not leave your AWS Account as part of this playground.

If you successfully sign-in, you will be presented with a page like this which shows there are three sample conversations with the three sample foundation model handlers. These were populated for you as part of the AWS CDK deploy that was done. Each one is with a different agent and are named as such.

Without any additional setup, you can chat with these three implementations and see how they each provide different experiences. The three handlers work as described, below:

1. Bedrock Direct Handler. This is a direct call to the bedrock claude-v2 model. When chatting with it, the client will trigger AppSync which then in turn invokes an async lambda function that queries Bedrock. This Lambda function will wait until a full response is generated, then provide the front-end with the update through a websocket subscription.
2. Bedrock Websocket Handler. This is similar to the direct handler, but instead of waiting for the full response, this handler will publish incremental updates to the websocket subscription. These updates are chunks of a few characters at a time, allowing the client to have immediate feedback when otherwise users would have to wait for the full response.
3. AppSync GraphQL Agent. This agent is exposed to another AppSync API which the agent can take actions against in response to user queries. The sample api represents a car dealership, so asking questions about inventory, salespeople, and orders can get you results pulled from associated data sources.

Feel free to play around with these handlers, chat with them, and see how they respond. Note that there will be some latency for the first few messages as lambda has a cold start time for new requests. To reduce this, lambda supports reserved concurrency which will allow you to keep an instance hot even when not in use.

Setting up Custom Agents

If you wish construct your own agents, there is an option for you to build your own agents on the playground. To do so, first you need to enable that option in the AWS CDK.

// Update the below line
const agentApi = buildAgentApi(this, { cognito, tables, enableConstructingAgents: false }

// Update to enable constructing agents
// This allows mutation options that any user can take to configure any agent they want
const agentApi = buildAgentApi(this, { cognito, tables, enableConstructingAgents: true }))

Next, re-deploy the AWS CDK

cd cdk-infrastructure
npm run deploy

And then configure the playground

cd playground
npm run configure

Then, after restarting the playground you should see a new wrench icon on the bottom right corner of the screen.

npm run start

Now, setup an agent to chat with. The demo repo has deployed a collection of AWS Lambda foundation model (FM) handers which you can configure into an agent to chat with. With Amazon Bedrock, you have the option to leverage fully managed agents for Amazon Bedrock. In the sample app, however, we use custom agents which can be configured to address your unique needs.

Clicking on the wrench icon in the bottom right switches you to the configuration mode for the sample app, and you should see the playground confirmation page.

Clicking on “New Agent” allows you to setup a new agent to chat with. Here you can enter a name, choose a handler (or enter a custom handler ARN) and set a system prompt for the agent. For this tutorial, we will build a new “Animal Facts” agent that makes use of a system prompt to give guidance to the agent.

We give the agent a system prompt like below:

You are a custom agent who can only talk about animal facts. Any conversation not about animal facts you will quickly redirect to animal facts. If the users asks a question not in this domain, tell them you do not know the answer and then provide a cool fact.

Below is the configuration page showing system prompt for animal facts agent.

Click “Create Agent” and the agent is now created and available to chat with. Clicking the chat bubble in the bottom right navigates you back to the chat panel where you can interact with this agent. Click “New Conversation” to bring up the list of agents, which now includes your new agent, then select the newly created agent.

Below is shown how to select the Animal fact agent to chat with.

We can now converse directly with the agent we built. We see below that it follows the system prompts and does not talk about cloud computing, but rather gives us a cool animal fact! Note that this is not a security tool to block access to specific content, it is a mechanism to help steer the agent’s responses towards what you wish it to do.

Below is shown a sample request response with the animal fact agent.

In another example, we can ask it to be an agent for AWS Appsync and keep the conversation directed as such. This is another system prompt we can use to build another agent.

You are an AI Agent for AWS AppSync.
Keep all conversations on-brand and refuse to answer anything that is not within this domain.

This will allow us to build an agent which responds to messages with the manner we request. Below is the conversation with this agent now with the AppSync prompt.

Connecting to Actions

Now lets create an action to enable an agent to connect to and take actions via an external API. As part of the AWS CDK infrastructure we deployed previously, we already set up a sample AWS AppSync GraphQL API that you can use to create an action. You can extend this API by adding additional datasources to it, or you can follow this pattern to bring your own API as well. For more details, see: Connect Amazon Bedrock to all your data with AWS AppSync and GraphQL.

Below is the configuration page shown again for the configured car-dealership agent. This action is configured for you as part of the CDK deployment, but you can choose to configure your own additional actions.

We can also create an agent to interface with this action. Again the configuration should already be present, but custom configurations are shown below. The “agent“ foundation model handler is setup to work with actions and should be used here.

If you start a new chat with this agent, you can now see the action defined on the right.

And if we ask it a question, the answer is able to be handled by the agent backing that execution through Claude! The agent itself is configured here using the reAct technique. In this prompting approach, FMs are provided data about the current conversation and what actions it can take, then the FM is asked to take steps to solve an abstract problem through a process of “Reason”, “Action”, and “Observation” steps.

In this way, we have created an AI agent that can converse with a user in natural language. Requests by the user are considered by the agent and then required actions are taken against the backend GraphQL API. Zero, one, or many actions may need to be invoked before the agent has gotten to a satisfying answer, at which point it returns the results to the user. A key thing to note here is that AWS AppSync is providing WebSocket subscriptions to enable users to see what is going on behind the scenes and visualize the actions performed by the agent in real time. For more details, see: Build a Real-time, WebSockets API for Amazon Bedrock.

Manual Invocation

As part of the sample car-dealership API, we have included a mock “send email” operation. However, the foundation model handlers we are using are explicitly denied access to this operation. They can discover that it exists because they can introspect the API, but they do not have permission to invoke it. This pattern can be followed to limit access to a destructive or dangerous action in your API that you don’t want an agent to have access to. If you ask the model to send an email, an action the permissions explicitly deny, the model will reply that it is unable to do so. You can, however, extend this pattern to enable a manual override. In our example, the application user is asked in the chat if they want to invoke the operation. If they do they can click the “invoke operation” link which will then use the sample app’s API key configuration to invoke it on behalf of the agent and then forward the results to it.

To get this set up, we first need to grab our API Key from the AWS AppSync console. Click on the “car dealer” API that should have been created for you by the AWS CDK.

Then, under settings, copy the API key that was also created through AWS CDK.

In the sample app, paste the API key under the credentials section of the action we created previously. It saves automatically.

Now, we can click the “click to invoke” on any operation the agent does, and the sample app will invoke the request for the agent with the API Key credentials!

Using the Code

The sample app is a fully featured React application which is a good jumping off point for those who want to build an production ready system like this for themselves. Lets take a quick look at how it is set up and how you can interface with it.

The playground is built with Amplify UI and utilizes their authenticator out of the box, meaning the entire website is secured with Amazon Cognito with this simple line below. All sign-in, signup, password reset, and confirmation logic is encapsulated here and is fully abstracted away.

export default withAuthenticator(PageWrapper)

Additionally the state of the site is managed through the recoil state management package. This interfaces with AWS AppSync hooks which will populate the website’s state store automatically as data is requested. See below:

export function renderConversation () {

    // Loads the conversationId from the react-router path
    const {conversationId} = useParams()
    
    // Hooks into recoil store, implictly loads objects if missing
    const conversation = useAgentApiConversation(conversationId)
    const agent = useAgentApiAgent(conversation.value.agent)

    // Easy abstraction around unloaded objects
    if (agentObject.isUnloaded() || agent.isUnloaded()){
        return <Loader />
    }
    
    // Many friendly Components build on Amplify UI
    return (
        <Container heading={`Chatting with '${agent.value.name}'`}>
            <ChatRendered
                 . . . 
            />
        </Container>  
    )
}

State is easily pulled from AWS AppSync through GraphQL queries built into hooks like so:

const listActionsQuery = new GraphqlQuery<GetActionsResponse>(`
    query MyQuery {
        listActions {
            id
            name
            resource
            type
        }
    }
`)

. . . 

listActionsQuery.invoke()
    .then(. . .)

Explore the full code for yourself at the demo repository here.

Conclusion

In this post we walked you through creating and then using a fullstack, web application powered by Amazon Bedrock. Learn more about how this application leverages AWS AppSync to both maintain an asynchronous connection between your frontend and Amazon Bedrock, and connect Amazon Bedrock to your data.