LLM Provost - Governance Proxy and Audit Ledger for LLM and MCP

LLM Provost is a governance proxy and audit ledger for MCP-mediated LLM interactions, designed for organizations under stringent regulatory oversight — including SEC Rule 17a-4, SEC Regulation SCI, SEC Rule 15c3-5, SEC Cybersecurity Disclosure Rules (Items 1.05 and 106), FINRA Rule 3110, FINRA Rule 3120, the Investment Advisers Act of 1940, CFTC Regulation 166.3, NIST AI RMF 1.0, NIST AI 600-1, SOC 2 Type II, HIPAA Security Rule (45 CFR §§ 164.302-318), HITECH Act, FDA AI/ML Guidance, CCPA/CPRA, FedRAMP, EU AI Act (Regulation (EU) 2024/1689), GDPR (EU 2016/679), MiFID II, MiFID II RTS 6, DORA (EU 2022/2554), ePrivacy Directive (2002/58/EC), EU Data Act (Regulation (EU) 2023/2854), EBA Guidelines on ICT and Security Risk Management, ESMA Supervisory Briefing on Algorithmic Trading, UK FCA SYSC, ISO/IEC 27001:2022, and ISO/IEC 42001:2023.

It logs everything between user, LLM, and MCP. Rules are applied for governance. Logs are retained per your compliance regime — SEC 17a-4 (3-6 years), MiFID II RTS 6 recordkeeping, DORA ICT incident reporting, HIPAA audit retention, and EU AI Act high-risk logging obligations.

LLM Provost acts as a mandatory policy and observability boundary in front of MCP servers and upstream model/tool APIs, giving you full control over every AI interaction. It runs in your own AWS account — you control the policy file and own the audit logs. A double-proxy architecture enforces policy at two boundaries: inbound (LLM clients to MCP servers) and outbound (MCP servers to upstream APIs), ensuring enforcement before any call leaves your trust boundary — meeting SEC Reg SCI, FINRA Rule 3120, and DORA operational resilience requirements.

Four-Point Audit Trail: Every governed request is captured across four events — client request entry, MCP server request exit, upstream response return, and MCP response return. Correlation fields (provost_user, provost_machine, provost_request_id) link all hops, satisfying SEC 17a-4, MiFID II RTS 6, EU AI Act Article 12, and ISO/IEC 42001:2023 transparency controls.

Core Governance Controls:

• Programmable guardrails to allow/block tool calls (SEC 15c3-5, CFTC 166.3)
• Per-tool rate limiting and token caps (FINRA 3110)
• Time-based access controls (MiFID II trading windows)
• Identity-rich audit logging with four-layer identity restoration (GDPR, HIPAA, SOC 2)
• LLM chat governance on non-MCP paths including PII filtering (GDPR Article 22, HIPAA, CCPA/CPRA)
• Per-MCP-server policy dispatch (NIST AI RMF Govern function)
• Alpaca trading controls: size/notional limits, symbol restrictions, forbidden endpoints, order replacement and close-position protections, optional trading windows (SEC 15c3-5, MiFID II RTS 6, ESMA, UK FCA SYSC 4-12)
• Hot-reload policy from rules.json with 10-second polling and automatic rollback (DORA change management, ISO/IEC 27001:2022)

Multiple LLM Backend Support: Route to OpenAI-compatible endpoints, Ollama, Amazon Bedrock, Anthropic, Gemini, and Responses API. Unknown backends return HTTP 404; misconfiguration fails closed with HTTP 500 — supporting DORA fail-safe principles and ISO/IEC 27001 availability.

Streaming Audit and Logging: Structured JSON logs capture request/response bodies for all governed paths. SSE streams are reconstructed into ordered semantic records up to 8 KiB. Credential-like fields are auto-redacted (GDPR data minimization, HIPAA safeguards). If the audit queue fills, processing fails closed (SEC Reg SCI, DORA). Fluent Bit ships to local files and optional S3 with Object Lock for WORM-compliant storage satisfying SEC Rule 17a-4(f).

Cognito Identity Integration: With LibreChat and Amazon Cognito, user identity is resolved from JWT claims — every audit record tied to an authenticated user (HIPAA access controls, GDPR accountability, SOC 2, FedRAMP).

Security Posture: no-new-privileges, dropped Linux capabilities, read-only root filesystems, tmpfs for transient data (ISO/IEC 27001:2022, EBA ICT guidelines, NIST AI RMF). Internal Docker network for MCP traffic. SHA256 digest pinning for all upstream images. Python dependencies installed with --require-hashes and scanned with pip-audit.

Human-in-the-Loop by Design: LLM Provost is not autonomous. Professionals use LibreChat to interact with AI; humans review, approve, and execute all decisions. Every step is logged, attributed, and immutable — satisfying FINRA 3110, EU AI Act Article 14, Investment Advisers Act of 1940 fiduciary duty, and NIST AI RMF accountability.

Deployment: Launch via AWS CloudFormation, set governance parameters, point your client at the endpoint. Open-source under AGPL-3.0 at https://github.com/CharmingSteve/llm-provost . AWS Marketplace AMI: https://aws.amazon.com/marketplace/pp/prodview-ouyql6wbwo6yg . Stateless AMI with reboot-persistent governance configuration also supported.