Amazon Bedrock FAQs

Agents for Amazon Bedrock are fully managed capabilities that make it easier for developers to create generative AI-based applications that can complete complex tasks for a wide range of use cases and deliver up-to-date answers based on proprietary knowledge sources. With just a few clicks, Agents for Amazon Bedrock automatically break down tasks and create an orchestration plan–without any manual coding. The agent securely connects to company data through an API, automatically converting data into a machine-readable format, and augmenting the request with relevant information to generate the most accurate response. Agents can then automatically call APIs to fulfill a user’s request. For example, a manufacturing company might want to develop a generative AI application that automates tracking inventory levels, sales data, supply chain information and can recommend optimal reorder points and quantities to maximize efficiency. As fully managed capabilities, Agents for Amazon Bedrock remove the undifferentiated lifting of managing system integration and infrastructure provisioning, allowing developers to use generative AI to its full extent throughout their organization.

You can securely connect FMs to your company data sources using Agents for Amazon Bedrock. With a knowledge base, you can use agents to give FMs in Amazon Bedrock access to additional data that helps the model generate more relevant, context-specific, and accurate responses without continually retraining the FM. Based on user input, agents identify the appropriate knowledge base, retrieve the relevant information, and add the information to the input prompt, giving the model more context information to generate a completion.

Agents for Amazon Bedrock can help you increase productivity, improve your customer service experience, or automate DevOps tasks.

With agents, developers have seamless support for monitoring, encryption, user permissions, and API invocation management without writing custom code. Agents for Amazon Bedrock automate the prompt engineering and orchestration of user-requested tasks. Developers can use the agent created prompt template as a baseline to further refine it for an enhanced user experience. They can update the user input, orchestration plan, and the FM response. With access to the prompt template developers have better control over the Agent orchestration.

With fully managed agents, you don’t have to worry about provisioning or managing infrastructure and can take applications to production faster.

Any customer content processed by Amazon Bedrock is encrypted and stored at rest in the AWS Region where you are using Amazon Bedrock.

No. Users inputs and model outputs are not shared with any model providers.

Amazon Bedrock offers several capabilities to support security and privacy requirements. Bedrock is in scope for common compliance standards such as Service and Organization Control (SOC), International Organization for Standardization (ISO), Health Insurance Portability and Accountability Act (HIPAA) eligible, and customers can use Bedrock in compliance with the General Data Protection Regulation (GDPR). Amazon Bedrock is included in the scope of the SOC 1, 2, 3 reports, allowing customers to gain insights into our security controls. We demonstrate compliance through extensive third-party audits of our AWS controls. Amazon Bedrock is one of the AWS services under ISO Compliance for the ISO 9001, ISO 27001, ISO 27017, ISO 27018, ISO 27701, ISO 22301, and ISO 20000 standards. Amazon Bedrock is CSA Security Trust Assurance and Risk (STAR) Level 2 certified, which validates the use of best practices and the security posture of AWS cloud offerings. With Amazon Bedrock, your content is not used to improve the base models and is not shared with any model providers. You can use AWS PrivateLink to establish private connectivity from your Amazon Virtual Private Cloud (VPC) to Amazon Bedrock, without having to expose your data to internet traffic.

No, AWS and the third-party model providers will not use any inputs to or outputs from Bedrock to train Amazon Titan or any third-party models.

Amazon Bedrock supports SDKs for runtime services. iOS and Android SDKs, as well as Java, JS, Python, CLI, .Net, Ruby, PHP, Go, and CPP support both text and speech input.

Streaming is supported on all the SDKs.

Please see the Amazon Bedrock Pricing Page for current pricing information.

Depending on your AWS support contract, Amazon Bedrock is supported under Developer Support, Business Support and Enterprise Support plans.

You can use CloudWatch metrics to track the inputs and output token.

With Amazon Bedrock, you can privately customize FMs, retaining control over how your data is used and encrypted. Amazon Bedrock makes a separate copy of the base foundational model and trains this private copy of the model. Your data including prompts, information used to supplement a prompt, FM responses, and customized FMs remain in the Region where the API call is processed.

When you’re fine tuning a model, your data is never exposed to the public internet, never leaves the AWS network, is securely transferred through your VPC, and is encrypted in transit and at rest. And, Bedrock enforces the same AWS access controls that you have with any of our other services.

We launched Continued Pre-training for Titan Text Express and Titan models on Amazon Bedrock; this will enable you to continue the pre-training on a Titan base model using large amounts of unlabeled data. This type of training will adapt the model from a general domain corpus to a more specific domain corpus such as medical, law, finance, etc. while still preserving most of the capabilities of the Titan base model. 

Typically, enterprises may want to build models for tasks in a specific domain. The base models may not be trained on the technical jargon used in that specific domain. Thus, fine-tuning the base model directly will require large amounts of labeled training records and a long training duration to get accurate results. To ease this burden, the customer can instead provide large amounts of unlabeled data for a Continued Pre-Training job. This job will adapt the Titan base model to the new domain. Then the customer may fine tune the newly pre-trained custom model to downstream tasks using significantly less labeled training records and with less training duration. 

Bedrock Continued Pre-training and Fine Tuning (FT) have very similar requirements. For this reason, we are choosing to create unified APIs that supports both CPT and FT. Unification of the APIs reduces the learning curve and will help customers use standard features such as CloudWatch Event Bridge to track long running jobs, S3 integration for fetching training data, Resource tags and Model encryption. 

Continued Pre-training helps you easily adapt the Titan models to your domain specific data while still preserving base functionality of the Titan models. To create a Continued Pre-training job, navigate to the Bedrock Console and click on ‘Custom Models’. You will navigate to the custom model page that has two tabs: Models and Training jobs. Both tabs provide a drop-down on the right called as “Customize Model”. Select “Continued Pre-training” from the Customize Model drop-down to navigate to the “Create Continued pre-training job “ screen. You will provide the source model, name, model encryption, input data, hyper-parameters and output data. Additionally, you can provide Tags along with details about IAM roles and resource policies for the job.

Exclusive to Amazon Bedrock, the Amazon Titan family of models incorporates Amazon’s 25 years of experience innovating with AI and machine learning across its business. Amazon Titan foundation models (FMs) provide customers with a breadth of high-performing image, multimodal, and text model choices, via a fully managed API. Amazon Titan models are created by AWS and pretrained on large datasets, making them powerful, general-purpose models built to support a variety of use cases, while also supporting the responsible use of AI. Use them as is or privately customize them with your own data.

To learn more about data processed to develop and train Amazon Titan FMs, visit the Amazon Titan Model Training & Privacy page.

Supported data formats include .pdf, .txt, .md, .html, .doc and .docx, .csv, .xls, and .xlsx files. Files must be uploaded to Amazon S3. Simply point to the location of your data in Amazon S3, and Knowledge Bases for Amazon Bedrock takes care of the entire ingestion workflow into your vector database.

Knowledge Bases for Amazon Bedrock provides three options to chunk text before converting it to embeddings. 

1.  Default option: Knowledge Bases for Amazon Bedrock automatically splits your document into chunks each containing 200 tokens ensuring that a sentence is not broken in the middle. If a document contains less than 200 tokens, then it is not split any further. An overlap of 20% of tokens is maintained between two consecutive chunks.

2.  Fixed size chunking: In this option, you can specify the maximum number of tokens per chunk and the overlap percentage between chunks for Knowledge Bases for Amazon Bedrock to automatically split your document into chunks ensuring that a sentence is not broken in the middle. 

3.  Create one embedding per document option: Amazon Bedrock creates one embedding per document. This option is suitable if you have pre-processed your documents by splitting them into separate files and do not want Bedrock to further chunk your documents.
 

At present, Knowledge Bases for Amazon Bedrock uses the latest version of the Titan Text Embeddings model available in Amazon Bedrock. Titan Text Embeddings model supports 8K tokens and 25+ languages and creates an embeddings of 1,536 dimension size. 

Knowledge Bases for Amazon Bedrock takes care of the entire ingestion workflow of converting your documents into embeddings (vector) and storing the embeddings in a specialized vector database. Knowledge Bases for Amazon Bedrock supports popular databases for vector storage, including vector engine for Amazon OpenSearch Serverless, Pinecone, Redis Enterprise Cloud, Amazon Aurora (coming soon), and MongoDB (coming soon). If you do not have an existing vector database, Amazon Bedrock creates an Amazon OpenSearch Serverless vector store for you.

Depending on your use case, you can use Amazon EventBridge to create a periodic or event driven sync between Amazon S3 to Knowledge Bases for Amazon Bedrock.

Model Evaluation on Amazon Bedrock allows you to evaluate, compare, and select the best foundation model for your use case in just a few clicks. Amazon Bedrock offers a choice of automatic evaluation and human evaluation. You can use automatic evaluation with predefined metrics such as accuracy, robustness, and toxicity. You can use human evaluation workflows for subjective or custom metrics such as friendliness, style, and alignment to brand voice. For human evaluation, you can leverage your in-house employees or an AWS-managed team as reviewers. Model evaluation provides built-in curated datasets or you can bring your own datasets.

You can evaluate variety of predefined metrics such as accuracy, robustness, and toxicity using automatic evaluations. You can also use human evaluation workflows for subjective or custom metrics, such as friendliness, relevance, style, and alignment to brand voice.

Automatic evaluations allow you to quickly narrow down the list of available FMs against standard criteria (such as accuracy, toxicity and robustness). Human-based evaluations are often used to evaluate more nuanced or subjective criteria which require human judgment and where automatic evaluations might not exist (such as brand voice, creative intent, friendliness).

You can quickly evaluate Bedrock models for metrics such as accuracy, robustness, and toxicity by leveraging curated built-in datasets, or by bringing your own prompt data sets. After your prompt datasets are sent to Amazon Bedrock models for inference, the model responses are scored with evaluation algorithms for each dimension. The backend engine aggregates individual prompt response scores into summary scores and presents them through easy-to-understand visual reports.

Amazon Bedrock allows you to set up human review workflows with a few clicks and bring your in-house employees or leverage an expert AWS-managed team to evaluate models. Through Amazon Bedrock’s intuitive interface, humans can review and give feedback on model responses by clicking thumbs up/down, rating on a scale of 1-5, choosing the best of multiple responses, or ranking prompts. For example, a team member can be shown how two models respond to the same prompt, and then be asked to select the model that shows more accurate, relevant, or stylistic outputs. You can specify the evaluation criteria that matter to you, simply by customizing the instructions and buttons to appear on the evaluation UI for your team. You can also provide detailed instructions with examples and the overall goal of model evaluation, so they can align their work accordingly. This method is useful to evaluate subjective criteria that require human judgement, or more nuanced subject matter expertise, and cannot be easily judged by automatic evaluations.

Guardrails for Amazon Bedrock enables you to implement safeguards for your generative AI applications based on your use cases and responsible AI policies. Guardrails helps control the interaction between users and Foundation Models (FMs) by filtering undesirable and harmful content and will soon redact personally identifiable information (PII), enhancing content safety and privacy in generative AI applications. You can create multiple guardrails with different configurations tailored to specific use cases. Additionally, you can continuously monitor and analyze user inputs and FM responses that may violate customer-defined policies in the guardrails.

Guardrails allows customers to define a set of policies to help safeguard your generative AI applications. You can configure the following policies in a guardrail.

• Denied Topics – You can define a set of topics that are undesirable in the context of your application. For example, an online banking assistant can be designed to refrain from providing investment advice.
• Content Filters – You can configure thresholds to filter harmful content across hate, insults, sexual and violence categories.
• Word Filters (Coming Soon) – You can define a set of words to block in user inputs and FM-generated responses.
• PII Redaction (Coming Soon) – You can select a set of PII that can be redacted in FM-generated responses. Based on the use case, you can also block a user input if it contains PII.

Guardrails can be used with all large language models available on Amazon Bedrock, including Titan, Anthropic Claude, Meta Llama 2, AI21 Jurassic and Cohere Command FMs. It can also be used with fine-tuned FMs as well as Agents for Amazon Bedrock.

AWS offers an uncapped intellectual property (IP) indemnity for copyright claims arising from generative output of the following generally available Amazon generative AI services: Amazon Titan models, Amazon CodeWhisperer Professional, and other Services listed in Section 50.10 of the Service Terms (the “Indemnified Generative AI Services”). This means that customers are protected from third-party claims alleging copyright infringement by the output generated by the Indemnified Generative AI Services in response to inputs or other data provided by the customer. Customers must also use the Services responsibly, such as not inputting infringing data or disabling a Service’s filtering features.

In addition, our standard IP indemnity for use of the Services protects customers from third-party claims alleging IP infringement (including copyright claims) by the Services and the data used to train them.