Distributed generative AI for government

Your agency runs on data such as case files, claims, clinical records, procurement, telemetry, and mission logs. A large language model (LLM) that can reason across that data is transformative for citizen services and mission velocity. The instinct is to consolidate: Pull the data into a unified warehouse so the model can see it. This instinct produces the siloed generative AI deployments now appearing across the public sector.

Each solves a single slice because consolidation across the organization is too expensive, too slow, and too risky. A distributed approach, one that brings generative AI to the data rather than the reverse, is achievable today using Amazon Web Services (AWS) solutions such as Amazon Bedrock for orchestration, Amazon Neptune for data lineage, and AWS Identity and Access Management (IAM) for source-point security enforcement.

You invest in generative AI to gain organizational insight and end up with point solutions that can’t see across the organization.

The cost of consolidation

The economic cost of consolidation is a recurring one. Moving data into a centralized warehouse incurs egress fees, duplicated storage, and Extract, Transform, and Load pipeline maintenance for every source. Each new dataset adds to your bill. The data management burden compounds: lineage breaks, consumers query the copy instead of the source, and freshness drifts until the warehouse and operational systems disagree on the same questions.

The access control cost is most acute: Copying data breaks row-level security rules and tag-based restrictions. The pragmatic response is to feed generative AI only the least-sensitive data the consolidation can safely include, which produces the least-valuable insights. Your data already lives where it needs to be. The challenge is to bring the generative AI to it.

In this post, you’ll learn how you can use six architectural patterns to deploy generative AI across your data without the cost and risk of consolidation. Central to these patterns is Tool-Object Level Access Protocol (TOLAP), an open security protocol built on AWS that enforces fine-grained access control at the data source using AWS IAM signing and Amazon Bedrock Agents so unauthorized data never enters the AI’s context window.

From pattern to platform: The Prism prototype

Government agencies need organization-wide AI insight, but their data is scattered across clouds, mission systems, and security boundaries that make consolidation impractical. Prism solves this by sending intelligent agents to where the data already lives, delivering answers without ever moving sensitive records out of their authorized environments.

Prism is a prototype built on AWS that demonstrates how generative AI can operate across data without consolidation. It uses Amazon Bedrock for orchestration, Amazon Neptune for data lineage, and Amazon Bedrock AgentCore Code Interpreter for more secure content generation. TOLAP is the open security protocol at Prism’s core, enforcing access control inside each tool call using AWS identity and signing.

Prism implements all six of the architectural patterns discussed in this article. Its purpose is to make the architecture tangible. Each pattern can be adopted independently. Prism is one shape they take when assembled. The solution features the following:

1. Distributed-by-design across AWS, Azure, and Google Cloud Platform. Identity on each cloud (AWS IAM roles, managed identity, workload identity federation). A central main account orchestrates; per-tenant worker accounts execute against the data wherever it lives.
2. Twenty-four source types: 11 database engines, six API protocols, five knowledge base platforms, three object stores, all mediated by three TOLAP secure tool plugins.
3. TOLAP implemented with signed security contexts, most-restrictive-wins policy merge, and column/row/field enforcement at the tool boundary.
4. AI-generated compliance checklists rendered alongside conversations in real time with configurable enforcement modes.
5. LLM-assisted metadata enrichment with cross-source lineage stored in Amazon Neptune and traversed during query planning.
6. Dynamic content creation (PDF, PowerPoint, DOCX, XLSX) in Amazon Bedrock AgentCore Code Interpreter, a managed, network-isolated sandbox receiving only TOLAP-filtered data.

Prism and TOLAP

The diagram in this section illustrates Prism’s end-to-end query flow across multiple cloud and on-premises environments. The core principle: Bring the intelligence to the data instead of bringing the data to a central lake.

In the diagram, a user describes what they need using natural language. Prism automatically identifies the right data sources, applies TOLAP security and governance, and dispatches processing tasks to wherever the data resides, whether that’s AWS, Google Cloud, Azure, or on premises. Each environment executes locally; data never crosses cloud boundaries. Only filtered, policy-compliant results return to the aggregation layer, which synthesizes them into action-ready summaries, reports, and visualizations.

TOLAP spans the entire pipeline, enforcing access control and audit requirements regardless of where data is processed.

The diagram shows how Prism routes a user’s natural-language request across multiple cloud environments and on-premises systems, returning only authorized, policy-compliant results without moving the underlying data:

Figure 1: Prism multi-cloud intelligence. Data never moves, only the answer does

The architectural reframe: Send intelligent agents to where the data already lives across cloud providers, mission boundaries, and source types, and enforce access policy at the point where the data is read. Reconverge the results centrally. The data never leaves its system of record. The systems that already enforce policies continue to honor them.

Six patterns make this architecture work. Each is independently adoptable. Together, they form a complete stack.

The following diagram illustrates how these six patterns layer together, from persona-level governance at the top through TOLAP enforcement to the distributed execution substrate that spans multiple clouds and data sources:

Figure 2: Organization-level generative AI architecture. This is the six-pattern governance stack

Pattern 1: The gated persona pattern

Open-ended chat is a governance liability. A general-purpose assistant given access to data will go off-script, reach for sources outside scope, generate queries that violate cost assumptions, and produce output that bypasses documentation standards. In a public sector environment, unbounded behavior is incompatible with mission.

The gated persona pattern bounds behavior. Each persona is a configuration: a role, a scope of data sources, allowed query types, expected output formats, and a structured compliance checklist generated by an LLM from the persona’s metadata.

Each checklist item declares an enforcement stage (pre-query, during query, post-query), a validation method, and a required or advisory flag. Enforcement mode is configurable per persona: advisory, blocking, audit, or disabled. The checklist surfaces alongside the conversation so users see what has passed and what’s pending. The persona isn’t a user interface (UI) choice, it’s the governance contract the agent operates under.

Pattern 2: TOLAP

Most generative AI authorization is bolted on at the wrong layer: The API gateway authenticates the request, the application validates the role, and the agent is then trusted to respect those decisions. But by the time the agent pulls results, it has accessed records and fields the user was never authorized to access.

TOLAP moves enforcement inside the tool at the point where data crosses from the source into the agent’s view: The agent calls a tool. The tool resolves the user’s effective policy from a signed security context, rewrites queries to inject row filters and column masks, restricts available endpoints, filters knowledge base results by tag, and returns only authorized data. The agent has nothing to compromise because unauthorized data never enters its context window.

The following diagram traces this authorization flow from the initial user request through policy resolution, credential validation, and more secure tool wrapping to the data source, showing how unauthorized data is excluded before it can reach the agent:

Figure 3: TOLAP authorization flow. This is source-point enforcement

A TOLAP-compliant system has five components:

1. Security context – A signed, time-bound, tamper-proof container that carries the user’s effective policy from the trusted authority into the tool execution environment
2. Security profiles – Declarative, reusable policy definitions, scoped per source category
3. Policy resolution engine – Computes the effective policy for a user by merging all assigned profiles using a most-restrictive-wins strategy
4. More secure tool wrappers – The tools themselves, each enforcing the resolved policy at execution time: query rewriting, column masking, result filtering, rate limiting, schema redaction
5. Secure tool factory – Instantiates tools with the correct security context for each request, handling credential resolution and source configuration

By design, this architecture reduces the attack surface by surviving prompt injection, jailbreak attempts, and unintended tool composition because the unauthorized data is never returned. This isn’t a claim that TOLAP eliminates all risk. It’s a claim that the attack surface is architecturally smaller.

Pattern 3: Connection breadth through TOLAP extensibility

Public sector data lives in relational databases, analytics warehouses, Representational State Transfer (REST), GraphQL, Fast Healthcare Interoperability Resources (FHIR), gRPC APIs, vector knowledge bases, and object stores across multiple clouds.

TOLAP makes breadth tractable because the protocol is category-aware but uniformly shaped. Adding a new connection means writing a more secure tool wrapper that interprets the relevant policy fields, not redesigning the authorization layer.

Pattern 4: Data normalization – LLM first, override second

Data rarely arrives normalized. The same entity appears under different identifiers across systems. LLMs are remarkably good at inferring equivalences, such as recognizing that “VA” and “U.S. Department of Veterans Affairs” are the same entity, or that dob and date_of_birth are the same field.

For relationships that aren’t obvious or are compliance-critical, data owners explicitly define overrides stored as graph relationships and applied at query planning time. LLM first means the solution delivers value without an army of data engineers. Override second means owners can pin what matters as the solution matures. It’s the safety net under the model, not the other way around.

Pattern 5: Metadata enrichment for confident discovery

A generative AI solution can’t answer questions about data it can’t find. This pattern gives data owners LLM-assisted tools to enrich metadata (descriptions, classifications, sensitivity flags) at every level. The solution vectorizes enriched metadata for semantic discovery. Metadata is the substrate of trust. Enrich it or guess.

Pattern 6: Dynamic content creation in a managed sandbox

Insights aren’t always tables. Agents generate PDF reports, PowerPoint decks, Word documents, and Excel workbooks by executing Python in Amazon Bedrock AgentCore Code Interpreter, a managed, network-isolated sandbox.

The sandbox receives only TOLAP-filtered data, so the principle holds: Unauthorized data never reaches the content generation layer. The user receives an artifact that reflects only what they’re permitted to see.

What this means for public sector leaders

There are three questions worth bringing to your architects:

1. Where is policy enforced in your generative AI architecture?

If the answer is “at the API gateway” or “in the agent prompt,” the architecture can’t scale to organization-wide data with strict access control. Source-point enforcement is the only model that survives prompt injection across heterogeneous source types.

2. What’s the unit of governance for your generative AI deployments?

If the unit is the model, behavior drifts across users and tasks. If the unit is the persona, which is a role-bound configuration with an enforced compliance checklist, behavior is auditable and adjustable without touching the underlying model.

3. Are you consolidating data to feed generative AI, or distributing generative AI to where the data lives?

Consolidation is the path of least architectural resistance and greatest long-term cost.

These patterns are an open architecture. Prism is one reference implementation. Evaluate which of the six patterns address your environment’s specific failure modes, adopt them independently, and measure whether insight improves without the cost and risk of consolidation.

Conclusion

Distributed generative AI isn’t a future aspiration, it’s an available architecture. The six patterns described in this post mean you can query data in place; enforce policy at the source; normalize entities without a data engineering army; and deliver governed, auditable AI output to any persona in your organization.

Together, they mean you don’t have to choose between organization-wide AI insight and the security, compliance, and cost discipline your mission demands.

To get started, evaluate which of the six patterns address your environment’s specific failure modes. Each pattern is independently adoptable. Start with the one that removes the biggest blocker for your agency. The Prism prototype provides a concrete reference implementation you can examine, adapt, and build on.

For more information about Amazon Bedrock, distributed AI architectures, and AWS solutions for government and public sector, visit the AWS Public Sector Blog.

For a deeper dive into how TOLAP closes the data-object security gap in AI agent architectures, check out the TOLAP: Closing the data-object security gap in AI agent architectures.