Connect Amazon Bedrock to your data with AWS AppSync and GraphQL

This post describes how you can leverage AWS AppSync and GraphQL APIs to seamlessly connect your Amazon Bedrock FMs and Agents to both public APIs and to your private APIs and databases. Amazon Bedrock is a generative AI service, and is the easiest way to build and scale generative AI applications with foundation models (FMs). With Amazon Bedrock’s comprehensive capabilities, you can easily experiment with a variety of top FMs, privately customize them with your data using techniques such as fine-tuning and retrieval augmented generation (RAG), and create managed agents that execute complex business tasks without writing any code. AWS AppSync is the easiest way to build an API that combines multiple microservice APIs and databases into a single, self-documenting, and introspectable API endpoint.

Exposing Your Data to Generative AI Presents New Challenges

The kinds of new applications you can build leveraging the capabilities of generative AI and FMs expands by providing FMs governed and controlled access to your private data. However, granting FMs access to private data gives rise to a new set of challenges, including:

• APIs provided to FMs need to be described in natural language, so FMs can understand actions they can perform, how they can be invoked, and what data they will return.
• FMs interfacing with private APIs require detailed error messages and an definition of valid inputs that they can use to make sure invocations they make are mapped to valid requests.
• FM access to private APIs should be tightly controlled to prevent FMs from taking actions that are unintended, destructive, or malicious.
• Actions taken by FMs and their agents need to be monitored and logged across many diverse systems and data stores.

There are different ways to connect FMs to enterprise data. Amazon Bedrock Agents, for example, provide an easy way to connect an FM to a well documented REST API. In this post, however, we will explore the benefits of using a custom FM agent and GraphQL APIs to connect your FMs to private data. With AWS AppSync and GraphQL you can provide FMs access to all your private data via a single, self-documenting, and machine-introspectable API. This makes it easy for FMs to access, understand, and interact with available enterprise data. Routing FM access to private data through a single GraphQL API endpoint also gives you a simple way to secure, govern, and observe FM actions. In this post we will describe an architecture, backed by GraphQL and AWS AppSync, which can act as the the connective layer between your FMs and your data. But, first, let’s explore how FMs work with external data.

How FMs Can Use External Data

The pace of progress in artificial intelligence in recent months and years has been staggering. Models have continued to get more powerful and pushed what they can do to new heights. Explorations into how we ask models questions (prompt engineering) has resulted in novel techniques like those captured in the reAct paper. In this prompting approach, FMs are provided data about the current conversation and what actions it can take, then the FM is asked to take steps to solve an abstract problem through a process of “Reason”, “Action”, and “Observation” steps.

Let’s see an example visualization of this prompting approach. In the illustration, below, the orange highlights are generated by the FM.

1. First, and key for all prompting approaches, a description of what the FM is asked to do and how it should act is provided. The FM is also told what actions it can take. For brevity the full description of the action in question is not included, but the FM would need a detailed explanation of what operations it has access to and how each of them can be invoked. In the working example we explore later, we provide a more detailed description of what the queryDatastore operation does, in what format it wants data, and what types are available.
2. The FM is asked to provide a “Reason,” which will be a natural language description of what the FM thinks about the current state and what should be considered as next steps.
3. Then the FM is asked for an “Action,” or the specific action the FM wishes to take.
4. FM generation pauses at this step and invokes the given action the FM requested. This action could be a database query, executing a workflow, interfacing with a vector store, or really anything else it may need to do. The output is provided back into the conversation history as the “Observation” of the action.
5. The FM then has a chance to respond again. In the above example it indicates it knows the answer now, but if the observation resulted in an unexpected result or did not provide the full picture, then this provides an opportunity to reason about what other data it needs.
6. The FM, having gotten to an answer, asks to invoke the “submit answer” tool and thus we stop the process and record this as the answer.

How GraphQL and AWS AppSync can connect FMs to private data

AWS AppSync enables developers to create secure, type safe, and flexible GraphQL APIs, providing a schema to parse, type check, authenticate, and resolve incoming requests to read and manipulate data against your constructed data graph. In this way, AWS AppSync sits as a secure and type-safe layer on top of your data. Permissions can be configured for each field of each type of data you define regardless of what data store or stores the data lives in. Types can be marked as optional and include sub types and recursive references. All of this configuration allows you to create a highly flexible data graph with an easy to use and descriptive query language that your FMs can leverage to interact efficiently with your private data.

Schema Definition as the Action Definition

AWS AppSync natively operates on a GraphQL Schema that defines what your API does in a clearly defined structure. For illustration, this example describes a car dealership API to interface with. A partial schema for such an API in AWS AppSync would look something like this:

type Car {
  id: ID!
  model: String!
  make: String!
  year: Int!
  color: String!
  price: Int!
}

type Query {
  # Gets all the cars for sale in your dealership
  listCars: [Car!]!
}

A production schema may provide more types, include mutation operations, and have parameters expected for field invocation. In the above GraphQL schema, the definition makes it clear that a query request to get all the colors, years, and prices of your cars can be written like so:

query {
    listCars {
        color
        year
        price
    }
}

Comments are even part of the schema definition itself and can provide guidance to the FM on why each field exists or what use each one has. The best part is that GraphQL has a special introspection operation which allows FMs to query for the schema definition as needed, with no additional configuration required.

Simply put, because GraphQL API’s are designed to be human readable they are also FM readable. A simple introspection query is enough for an FM to understand your private APIs and build queries and mutations against them.

Built-in Type Safety and Natural Language Errors

AWS AppSync securely parses your query against the schema when the request is received. If you define a field as an integer, then AWS AppSync will not allow it to be invoked if it is anything else, removing the need for special parsing of the FM’s query.

This also means that requests that are malformed or typed incorrectly will return clearly articulated error messages explaining what went wrong. If you ask for a value that does not exist, AWS AppSync returns an FM readable error message, one that the FM can use to correct its error.

query {
  listCars {
    colors
    id
  }
}

Validation error of type FieldUndefined: 
Field 'colors' in type 'Car' is undefined

These kinds of natural language error messages help push the FM to the right query and enable it to unstick itself when it makes a mistake.

Authenticated By Default

AWS AppSync provides authentication on a per-field basis and has support for AWS Identity and Access Management (IAM), Amazon Cognito, OIDC, API Keys, and even custom auth logic through AWS Lambda. If you wish to prevent or restrict the FM from invoking a field or viewing certain data, you have all the options needed to make that happen implicitly while allowing other users to still use the API as normal.

If you want to prevent a certain operation from being invoked, you can deny permission through any of the authentication modes AWS AppSync provides. This can give an FM insights into what actions could be taken without opening it up to your entire back end data graph.

mutation {
  sendEmail(
    title: "Sample Email", 
    to: "joe.schmo@amazon.com"
    body: "Hey this is a test of an AppSync send-email resolver!"
  )
}

Not Authorized to access sendEmail on type Mutation

Because AWS AppSync GraphQL APIs can combine one or more data stores, this means that in one place you can configure and govern control over what operations the FM can take against your private data, and which operations it can’t. The FM can then relay error messages like this to the application user or choose to try a different approach that may not cause the error. You can also choose to use the permissions of the invoking user to define and limit what actions the FM can take. In this way, you can limit exposure to confused deputy attacks because the FM has no elevated permissions beyond those of the invoking user.

Configurable Logging Solutions

AWS AppSync also comes with customizable and detailed logging levels, including metrics, log streams from your API, and custom logging options written into your resolver logic. This provides clear visibility into which queries, fields, resolvers, and data the FM was able to access or was denied from accessing. This gives the developer full visibility and auditing for FM actions.

Example data flow with AWS AppSync and GraphQL

The below diagram is the path a sample request make take when an FM is asked to solve an abstract problem against your back-end APIs.

Step by step, the process is as follows:

1. The example above implements the reAct Solver using LangChain in a Lambda function, but this architecture is usable for any compute service of choice. Lambda was chosen for simplicity.
2. As AWS AppSync natively exposes the schema to users through introspection queries, we can use this schema as our action space definition in our FM prompt. No prompt engineering is required, but you can add descriptions to our GraphQL fields natively in the schema to help smooth over any non-intuitive routes in the API.
3. The first invocation of our FM model. The FM will be prompted to provide both the reason and the action output step. Stop words can be used to prevent it from responding outside of this result.
4. With an AWS AppSync realtime subscription, the invocation reasons and actions are immediately available to the end user application, and enable a user to interact with the intermediate actions of the FM agent in realtime as they run. Without this ability, the user may be stuck waiting for a result for minutes with no feedback.
5. With the generated action, the AWS AppSync API is invoked to get the result asked for by the FM. This is type safe, backed by IAM security, and includes Amazon CloudWatch logging for observability.
6. Data is ultimately pulled from connected AWS AppSync data sources. AWS AppSync natively supports connections to Amazon DynamoDB, Amazon Relational Database Service (Amazon RDS), AWS Lambda and many other AWS services. These data connections are made through JavaScript resolvers that can provide additional logic for additional authentication and data transformation needs.
7. The FM model now has the result of its query and is prompted a second time. The FM is allowed to either output a “final answer” result which is then exposed to the user or an additional “reason + action” result which then may invoke a follow-up query to the data stores.

Conclusion

This article described how AWS AppSync and GraphQL is a natural fit for a connective layer between FMs and your private data. In this way, you can leverage Appsync to provide FMs with the data they need to expand beyond their native capabilities.

This post was written by Eric Robertson, AWS SDE