Building a RAG architecture on Amazon Bedrock to support CJIS compliance

Analysts manually reviewing virtual reams of sensitive documents, looking for one key detail, need a more efficient way to search. They know generative AI must be able to do the job, but what they don’t know is whether they can apply generative AI to law enforcement data. You can, and Amazon Web Services (AWS) offers an efficient option with managed services.

This post walks through a reference architecture for a Retrieval Augmented Generation (RAG)-based intelligence retrieval tool that law enforcement agencies can deploy so analysts can ask natural language questions against voluminous evidence, case files, investigative reports, and CJIS-classified records. The architecture is deployable in commercial AWS Regions and AWS GovCloud (US).

What we’re building

Consider a municipal intelligence unit that needs a way to ingest Criminal Justice Information Services (CJIS)-classified records and internal departmental documents. They need personnel to be able to query with natural language. Responses must cite sources that reference the original document. The solution must comply with FBI CJIS Security Policy v6.1 and must come at a cost that is accessible to municipal IT budgets. And intelligence units need all this without compromising security.

The data could include CJIS records, law enforcement sensitive (LES) documents, investigative reports, and intelligence briefings. All this data requires need-to-know access controls, encryption at rest and in transit, and audit logging.

CJIS compliance in commercial Regions

The CJIS Security Policy v6.1 (June 2026, building on v6.0 released December 2024) doesn’t mandate a specific vendor, cloud provider, Federal Risk and Authorization Management Program (FedRAMP) authorization level, or deployment model. Commercial Regions and AWS GovCloud (US) can support CJIS workloads.

There are two key dates to note. In August 2021, AWS announced that customers in all 50 states can host criminal justice information on AWS, after shifting from a personnel-security model with state-by-state background check addenda to a technical-controls model relying on customer managed encryption using AWS Nitro System and AWS Key Management Service (AWS KMS). In October 2023, AWS published updated guidance reaffirming support for CJIS workloads in both commercial Regions and AWS GovCloud (US).

Commercial Regions are viable for CJIS today due to several factors. AWS Nitro System and AWS KMS architecture eliminate AWS personnel access to unencrypted criminal justice information (CJI), which is the same encryption model as AWS GovCloud (US). Federal Information Processing Standard (FIPS) 140-3 endpoints are available in commercial Regions, and data residency is satisfied by US commercial Regions because data stays in US West (Oregon) – us-west-2.

When deciding between commercial Regions and AWS GovCloud (US), use the following metric:

• When AWS GovCloud (US) is the right choice – If your agency has overlapping compliance requirements, such as FedRAMP High, International Traffic in Arms Regulations (ITAR), or Department of Defense (DoD) cloud Impact Level (IL) 4 or IL 5, AWS GovCloud (US) provides a single environment that satisfies all of them. This same architecture deploys in AWS GovCloud (US) without modification.
• When commercial Regions are a good fit – For agencies where CJIS is the primary compliance driver and no overlapping federal mandate requires AWS GovCloud (US), commercial Regions offer standard pricing, broad service availability, and faster feature launches.

Architecture

This architecture uses US West (Oregon) – us-west-2 . The end-to-end flow is fully managed and serverless:

1. Ingestion – Authorized personnel upload records to a dedicated Amazon Simple Storage Service (Amazon S3) bucket with server-side encryption with AWS KMS (SSE-KMS) using customer managed keys.
2. Processing – Amazon Bedrock Managed Knowledge Base handles everything including chunking, embedding, and vector indexing. Parsing, embedding, and reranking are included at no additional cost.
3. Query – The analyst submits a natural language prompt.
4. Agentic retrieval – The agent calls AgenticRetrieveStream through the gateway. The managed knowledge base plans a retrieval strategy, executes multiple search passes (semantic and keyword), and streams back results alongside a synthesized, citation-backed answer in one tool call.
5. Response – The agent returns the knowledge base’s cited answer to the analyst. Because the knowledge base handles synthesis internally, the agent can pass the answer through directly or layer additional reasoning on top.

This architecture is shown in the following diagram.

Figure 1: RAG architecture on Amazon Bedrock with security controls designed to support CJIS compliance

MCP through Amazon Bedrock AgentCore Gateway

A key architectural decision in this design is how the knowledge base is exposed to consuming agents. By using Amazon Bedrock AgentCore Gateway with Model Context Protocol (MCP) rather than building a custom API integration, you get:

• Agent-agnostic access – Amazon Bedrock Agents, Strands Agents, and Kiro command line interface (CLI) can all query the same knowledge base without custom code
• Ability to swap without rearchitecting – Upgrade or replace the frontend agent without touching the retrieval backend
• Extensibility – Add more data sources later through the same MCP interface
• Built-in authorization – AWS Identity and Access Management (IAM) authorization at the gateway layer with AWS CloudTrail logging

Two retrieval modes are available through the gateway connector. This architecture uses AgenticRetrieveStream, in which the knowledge base plans a multistep retrieval strategy, runs multiple search passes, and returns a synthesized, citation-backed answer in one tool call. For more straightforward use cases or when you want full control over prompt engineering, Retrieve performs a single hybrid search and returns raw passages for your agent to synthesize independently.

Chunking strategy

Amazon Bedrock Managed Knowledge Base supports configurable fixed-size chunking, at a configurable size, defaulting to 300. For context continuity, there’s a 20 percent overlap between adjacent chunks. File name, date, and classification level are extracted as filterable metadata

I’d recommend increasing the default chunk size for investigative documents. 300 tokens is fine for short, structured records, but for long narrative reports, you’ll want larger chunks (500–800 tokens) so a paragraph about a subject’s known associates stays in one retrievable unit.

Semantic chunking is also an option, but you’d use a customer managed knowledge base with your own vector store instead. For most use cases, tuned fixed-size chunking with the managed knowledge base’s built-in hybrid search and reranking provides you with equivalent retrieval quality with less operational overhead.

Security architecture

This section provides prescriptive guidance for the controls that auditors prioritize in CJIS environments.

First, encryption controls:

• At rest –AWS KMS with customer managed keys, FIPS 140-3 Level 3 validated hardware security modules (HSMs)
• In transit –TLS 1.2+ with FIPS endpoints (AWS_USE_FIPS_ENDPOINT=true)
• S3 bucket policy enforces aws:SecureTransport, denying any plaintext requests.

For network isolation, we can configure zero internet egress with the following controls:

• Amazon Virtual Private Cloud (Amazon VPC) endpoints (AWS PrivateLink) for all services, including Amazon S3, Amazon Bedrock, AWS KMS, and AWS CloudTrail
• Security groups restrict traffic to known Classless Inter-Domain Routing (CIDR) ranges
• No public subnets—all resources in private subnets only
• FIPS endpoints for all API calls

To control access, we have:

• Dedicated IAM roles per component (such as ingestion and query) to maintain separation of duties
• AWS IAM Identity Center for analyst authentication
• Multi-factor authentication (MFA) required for all console access
• Condition keys restrict access to specific Amazon S3 prefixes and knowledge base resources
• Automatic session timeout after 30 minutes of inactivity

On top of the security controls, we have robust logging for details audits:

• CloudTrail captures 100 percent of API calls (such as Amazon Bedrock invocations, Amazon S3 access, AWS KMS operations)
• Amazon Bedrock model invocation logging records full prompt-response pairs
• Amazon S3 server access logging for record-level access tracking
• Log retention aligned with agency policy (typically 5–7 years for CJIS environments)

Document-level access control (ACL-aware retrieval)

Not every analyst should see every document. Case compartmentalization is a core requirement. Amazon Bedrock Managed Knowledge Base supports access control list (ACL)-aware retrieval, which is the ability to filter query results based on document-level permissions. When enabled, the knowledge base crawls document permissions from the data source. At query time, it filters results based on the user context you provide. For connectors that support it, the knowledge base makes a real-time call back to the data source to verify the user still has access.

Deny always overrides allow. If a user appears in both lists for a document, access is denied.

ACL awareness is not authorization.

The managed knowledge base provides filtering, not a security boundary. It doesn’t authenticate end users; your application is responsible for authenticating users before passing their identity to the knowledge base, confirming that the email address you pass matches the email in the data source and managing the email lifecycle.

The knowledge base fails closed. If ACL evaluation encounters an error, it returns zero results rather than leaking documents to unauthorized users. That’s the correct behavior for CJIS data. ACL updates are cached for up to 1 hour, so permission revocations might not take effect immediately. Plan your access-revocation procedures accordingly.

This reference architecture describes technical controls you can implement on AWS. Achieving CJIS compliance also requires organizational policies, personnel controls, and agreements between your agency and AWS. There is no central body that authorizes a system as CJIS-compliant. Compliance is assessed by your state’s CJIS Systems Agency (CSA). This post isn’t legal advice; consult your compliance team and legal counsel to validate this architecture against your specific requirements. Refer to the AWS Shared Responsibility Model and the AWS CJIS compliance page.

You can use a phased approach that validates retrieval quality before investing in a production UI. First, use the Amazon Bedrock console for testing with no development effort. You can move to an agent such as Kiro CLI for MCP testing with minimal development. For a production-grade implementation, I’d suggest an Amazon Bedrock agent through Amazon Bedrock AgentCore Gateway.

What it costs

For a proof of concept (POC)-scale deployment (approximately 5 GB of raw documents, 2,000 queries per month), a reasonable estimate gives us a monthly price under $200—a price accessible to agencies of any size.

Amazon Bedrock Managed Knowledge Base pricing is $5.00 per GB of raw data indexed per month, $4.00 per 1,000 AgenticRetrieveStream API calls (compared to $1.00 for standard Retrieve), and $0 for parsing, embeddings, and reranking (included).

Next steps

Generative AI for law enforcement intelligence is deployable today. It’s fully managed and in a standard commercial AWS Region with technical controls designed to support CJIS compliance, at a price point accessible to municipal agencies.

Agencies working toward CJIS compliance can use this architecture as a starting point for natural language retrieval. Whether you deploy in a commercial Region or in AWS GovCloud (US), the components and security controls are the same. The choice comes down to your agency’s full compliance picture. Consult your compliance team and state CSA to validate it against your specific requirements.

To get started, create a managed knowledge base in your AWS account today. You can have a working prototype in under 1 hour using the Amazon Bedrock console. For more information, refer to Amazon Bedrock Knowledge Bases and the Amazon Bedrock Managed Knowledge Base announcement. For more about CJIS compliance, read Continued innovation in CJIS compliance in both AWS GovCloud (US) and AWS US commercial Regions in the AWS Public Sector Blog.