CORTO’s billion-scale legal semantic search with Aurora PostgreSQL pgvector

When a legal team needs to find the right precedent, relevant clause, or key fact across decades of case files, every hour saved through semantic search is several hours reclaimed for strategic counsel and client service. CORTO’s AI-powered semantic search gives legal teams their time back, returning instant, accurate results from billions of documents using Amazon Aurora PostgreSQL and pgvector optimized for billion-scale vector search.

CORTO is an AI-powered legal technology platform serving 10,000+ law firms globally. It makes 2.5 billion documents spanning decades of legal knowledge instantly searchable through a production system managing 7.6 billion vectors in a 46 TB Aurora PostgreSQL cluster. CORTO chose Aurora PostgreSQL with pgvector over purpose-built vector databases, migrated to compact embeddings, and implemented multi-tenant partitioning with logical per-firm isolation. These decisions reduced storage costs while maintaining sub-second query performance. Using existing PostgreSQL expertise, CORTO transformed how legal teams access decades of precedent.

In this post, we describe how CORTO built and scaled this system, from the decision to use Aurora PostgreSQL with pgvector over purpose-built vector databases, through the embedding model strategy that drives cost and performance optimization, to the multi-tenant design that maintains logical data isolation across thousands of firms, and the operational insights gained at production scale.

Making decades of legal knowledge instantly accessible

Law firms accumulate decades of legal knowledge, from case precedents and contracts to opinions and filings, that legal teams must search and apply in near real time. CORTO set out to solve this at scale, with four core requirements:

1. Legacy data accessibility: Make decades of legal documents instantly searchable through AI-powered semantic search.
2. Multi-tenant architecture: Serve law firms with tenant isolation and compliance requirements.
3. Cost-efficient scale: Handle billions of documents while maintaining sustainable economics.
4. Near real-time performance: Support near real-time document ingestion and semantic search.

In 2023, the managed AI landscape was still early. Amazon Bedrock was in preview, and production-scale vector search patterns were not yet widely documented. Adopting a dedicated vector database would have meant asking CORTO’s team to learn an entirely new system from scratch, at the same time as building the application on top of it.

CORTO needed to reach production quickly, on infrastructure their engineering team already knew how to operate, while meeting a 99.99 percent availability service level agreement (SLA) and legal industry compliance requirements from day one.

Why Aurora PostgreSQL and pgvector?

CORTO’s engineering team evaluated multiple vector database options before selecting Aurora PostgreSQL with pgvector. The decision came down to two practical advantages:

First, the team’s deep PostgreSQL expertise meant they could move from prototype to production rapidly without rebuilding operational knowledge from scratch. Second, running vector search inside Aurora PostgreSQL avoided operating a separate vector store. Embeddings ran on the same managed platform, and relational filters like firm_id could be applied in the same query as similarity search.

Solution overview

CORTO’s production architecture is a multi-tenant system purpose-built for scale, performance, and cost efficiency, organized into three functional layers: an Integration Pillar for document ingestion, a Data Pillar for storage and retrieval, and an AI Layer for embedding and language model processing. The following diagram illustrates how CORTO ingests legal documents from clients, converts them into vector embeddings, stores them in Aurora PostgreSQL with pgvector, and serves them through the AI API.

Figure 1: CORTO’s multi-tenant architecture across the Integration, Data, and AI layers

Integration Pillar: This layer connects to external cloud storage providers where CORTO’s clients load their legal documents. CORTO uses Amazon Textract to extract documents, handling everything from native PDFs to scanned legacy files, staging the extracted content in Amazon Simple Storage Service (Amazon S3) as the central document repository. CORTO implements priority-based ingestion (detailed in the Document Processing Pipeline section) to balance real-time and batch processing. The system ingests important documents, such as newly uploaded files and active matter content, in near real-time, and processes non-critical documents in batch through Amazon Simple Queue Service (Amazon SQS) queues.

Data Pillar: CORTO stores the 384-dimension vectors in a dedicated Aurora PostgreSQL cluster with pgvector for fast semantic search, and stores document metadata and text chunks in Amazon DynamoDB for efficient ingestion and retrieval. This separation keeps Aurora PostgreSQL optimized purely for vector operations, while DynamoDB handles high-throughput retrieval of the actual document content that surfaces in search results. A separate Aurora PostgreSQL cluster serves as the AI memory layer, persisting conversational context and session state to support coherent, multi-turn legal research interactions.

AI Layer: From S3, CORTO passes documents to an Amazon Elastic Container Service (Amazon ECS)-based processing pipeline that performs chunking and generates vector embeddings using a self-hosted Nomic embedding model. At query time, the system’s AI API, also hosted on Amazon ECS, orchestrates retrieval across the Data Pillar stores to deliver contextually aware, matter-ready responses to external clients.

Today, the platform manages 7.6 billion vectors across 2.5 billion documents in a 46 TB Aurora PostgreSQL cluster deployed for Asia Pacific (APAC) customers, with two additional clusters serving Americas and EU customers.

Technical implementation journey

CORTO’s initial prototype was built in under two months, validating the core hypothesis that Aurora PostgreSQL with pgvector could handle legal document embeddings at scale. The early architecture was intentionally simple. An Aurora PostgreSQL cluster with pgvector. All embeddings and bulk text stored in a single table, OpenAI ADA embeddings at 1,536 dimensions, and basic multi-tenant partitioning. The prototype successfully proved the concept.

After validating the core concept with an early prototype using Aurora PostgreSQL with pgvector, CORTO’s team turned to scaling the system for production. Managing embeddings at scale, addressing architectural decisions, spanning storage design, ingestion infrastructure, document processing, and the embedding model itself, that would define the platform’s production foundation.

Separating vector storage from bulk text

In the prototype, vectors, document text, and metadata all coexisted in a single Aurora PostgreSQL table. This worked at small scale, but the schema design, not the platform, became the constraint as the dataset grew into the billions: storing bulk text alongside vectors in the same table meant every vector operation carried the I/O cost of rows it did not need. The team restructured storage around a separation of concerns: Aurora PostgreSQL with pgvector became exclusively responsible for vector embeddings and similarity search, Amazon DynamoDB took over document metadata and text chunks, and Amazon S3 handled full document archival. Each layer could now be independently scaled and optimized for its specific workload.

Self-hosted embedding infrastructure

Reliance on external embedding APIs had capped ingestion capacity across thousands of law firm tenants. Rate limits from third-party providers constrained how many documents CORTO could process simultaneously. The solution was to deploy self-hosted Nomic embedding infrastructure on managed compute, giving the team full control over ingestion concurrency, model versioning, cost predictability, and data residency. Eliminating this external dependency removed the primary ingestion bottleneck and enabled horizontal scaling based on demand, processing priority documents in near real-time while batching non-critical workloads during off-peak hours.

Multi-tenant vector search implementation

Serving law firms requires logical isolation per firm and regulatory compliance while maintaining high query performance. The team implemented a partitioned table with per-firm partial indexes for their multi-tenant law firm clients. CORTO hash-partitions the embeddings table by firm_id, with one partial HNSW index per firm (WHERE firm_id = N). When a firm runs a semantic search, the query filters on its own firm_id. Partition pruning routes the query to the single hash partition holding that firm’s rows, and that firm’s partial index serves the vector search. Because each query touches only one firm’s partial index, searches stay isolated per tenant. The per-query working set also stays small enough to remain in memory, so searches return in sub-second time.

This partial index is an HNSW index built on the half-precision (halfvec) cast of the vectors, which makes it roughly half the size of a full-precision index. At 384 dimensions, halfvec provides a 2x storage reduction with negligible recall loss, keeping per-firm indexes small enough to remain fully cache-resident even as firm document counts grow. To keep more vector data resident in cache as the dataset grows, CORTO runs the cluster on Aurora Optimized Reads instances, which extend the buffer cache to local NVMe SSD.

Scaling happens at the index layer. Each new firm adds one partial HNSW index while hash partitioning keeps the partition count fixed, so CORTO grows the tenant base by adding indexes rather than reworking the partitioning schema. Performance scales the same way. Because a query only ever traverses a single firm’s partial index, latency is bounded by that firm’s data size rather than the full dataset, keeping performance consistent as the platform grows.

Aligning the index boundary with the tenant boundary is the intended design pattern for multi-tenant workloads on pgvector rather than a workaround for dataset size. Because every legal search is inherently scoped to a single firm for compliance reasons, a per-firm index adds no restriction the application did not already require, and it converts a compliance constraint into a performance advantage. pgvector HNSW also serves large single-tenant indexes well; CORTO’s clusters hold 7.6 billion vectors in total, and partitioning by firm_id means no individual query needs to traverse all of them.

CORTO also separates workloads at the instance level. Ingestion runs against the Aurora writer while similarity searches are served from a read replica, so write-heavy embedding ingestion does not compete with read-heavy vector search, and read capacity can scale out by adding replicas.

-- Example schema (simplified for illustration)
CREATE TABLE embeddings (
    id SERIAL,
    firm_id INTEGER NOT NULL,
    vector vector(384),
    document_id VARCHAR(255),
    created_at TIMESTAMP DEFAULT NOW(),
    PRIMARY KEY (id, firm_id)
) PARTITION BY HASH (firm_id);

-- Create partition for a tenant
CREATE TABLE embeddings_p0 PARTITION OF embeddings
FOR VALUES WITH (MODULUS 4, REMAINDER 0);

-- Partial HNSW index with halfvec operator for efficient firm-level filtering
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_embeddings_vector_firm_1
ON embeddings_p0
USING hnsw ((vector::halfvec(384)) halfvec_ip_ops)
WITH (m = 16,ef_construction = 256)
WHERE firm_id = 1;

-- Verify HNSW partial index usage: partition pruning routes to embedding_p0, partial index scan on idx_embeddings_vector_firm_1
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, document_id,
       (vector::halfvec(384)) <#> '[-0.0382628,0.0731812,...,0.0825291]'::halfvec(384) AS similarity
FROM embeddings
WHERE firm_id = 1
ORDER BY (vector::halfvec(384)) <#> '[-0.0382628,0.0731812,...,0.0825291]'::halfvec(384)
LIMIT 10;

Limit (cost=442.35..466.15 rows=10 width=25) (actual time=0.778..0.795 rows=10 loops=1)
  Buffers: shared hit=758
  -> Index Scan using idx_embeddings_vector_firm_1 on embeddings_p0 embeddings (cost=442.35..24245.02 rows=10001 width=25) (actual time=0.777..0.793 rows=10 loops=1)
        Order By: ((vector)::halfvec(384) <#> '[-0.038269043,0.07318115,...,0.08251953]'::halfvec(384))
        Buffers: shared hit=758
Planning:
  Buffers: shared hit=169
Planning Time: 0.567 ms
Execution Time: 0.913 ms
(9 rows)

The 0.913 ms execution time with all buffer cache hits (shared hit=758) confirms the partial index strategy keeps each firm’s working set fully cache-resident, delivering sub-millisecond semantic search regardless of total platform scale.

Document processing and ingestion pipeline

Making 20–30 years of legacy documents accessible through AI-powered search presented a challenge unique to the legal domain. Legal archives span decades of inconsistent formatting, scanned images, degraded paper documents, and legacy file formats. CORTO addressed this with a tiered text extraction strategy that routes each document through the most cost-effective method that meets quality requirements: native extraction for modern digital files, Amazon Textract for scanned content, and advanced optical character recognition (OCR) for the most complex or degraded cases. This intelligent routing made it economically viable to process billions of legacy documents that would have been cost-prohibitive under a uniform approach.

Equally important was the end-to-end pipeline architecture, which follows four stages:

1. Ingestion, where documents arrive from client cloud storage through the Integration Pillar.
2. Extraction, where the pipeline processes content through the preceding tiered strategy.
3. Chunking and embedding, where the pipeline splits extracted text into semantically meaningful segments and converts them to 384-dimension vectors on self-hosted Nomic infrastructure.
4. Storage and indexing, where the pipeline writes vectors to the appropriate firm partition in Aurora PostgreSQL with HNSW index updates.

The ingestion pipeline uses a queue-based architecture with prioritization logic: it processes critical documents in near real-time and batches less urgent workloads during off-peak hours. Newly uploaded documents and active matter content flow through a priority queue for immediate processing, while the pipeline handles historical and archived documents in high-throughput background batches without impacting foreground performance.

Throughout the entire pipeline, from document upload through text extraction, embedding generation, and vector storage, CORTO maintains firm-level data separation, preserving logical multi-tenant isolation at every stage.

Figure 2: The document processing and ingestion pipeline stages

Embedding model migration

The final optimization addressed storage costs at their root through a deliberate model evaluation. The team benchmarked multiple embedding models against legal-specific retrieval tasks, measuring recall, precision, and latency at production scale. Three factors drove the final decision:

1. For legal semantic search, where queries match conceptual meaning rather than exact keywords, 384 dimensions captured sufficient semantic signal with no measurable recall degradation on their test set.
2. Self-hosting capability was critical for data residency compliance across APAC, EU, and Americas regions.
3. Deterministic outputs enabled reproducible search results, important for legal audit trails.

CORTO then migrated from OpenAI ADA (1,536 dimensions) to Nomic embeddings (384 dimensions). This 75 percent reduction in vector dimensions maintained retrieval quality on CORTO’s legal evaluation set with no measurable recall degradation. This architectural decision delivered cost savings that compound with every additional document ingested and every query served at scale.

Figure 3: Embedding model migration from 1,536 to 384 dimensions

Cost optimization strategies

The most impactful cost decision, migrating from OpenAI ADA to Nomic embeddings for a 75 percent reduction in storage requirements, is covered in the Embedding model migration section above. The following strategies address the remaining layers of the stack, keeping the platform economically sustainable as it scales.

Compute optimization: Amazon Elastic Compute Cloud (Amazon EC2) Spot Instances power non-critical workloads including embedding generation and document processing, reducing compute costs significantly compared to on-demand pricing. CORTO right-sizes instance types with continuous monitoring and adjustment based on actual workload patterns, while AWS Fargate resource tuning optimizes containerized embedding workflows.

Database optimization: A storage tiering strategy places hot vector data in Aurora PostgreSQL with Optimized Reads, warm text chunks and metadata in DynamoDB, and cold archival documents in S3. AWS Database Savings Plans address predictable Aurora workloads as the platform scales, and efficient HNSW indexing with table partitioning strategies minimize query compute overhead.

Text extraction optimization: A tiered extraction strategy routes each document through the most cost-effective method that meets quality requirements, with intelligent routing based on document complexity assessment before selecting the extraction tier. CORTO uses per-document-type cost tracking to continuously refine routing logic over time.

LLM cost management: CORTO routes each use case to the most cost-effective model for the task, with Amazon Bedrock and Claude serving the majority of inference. Batch processing handles non-time-sensitive operations to reduce real-time inference costs.

Operational efficiency: Priority-based ingestion reduces peak compute demand by batching non-urgent workloads during off-peak hours, while DevOps automation reduces on-call burden through comprehensive monitoring and automated remediation.

Operational insights

Observability journey: As CORTO’s document processing pipeline scaled to billions of vectors, effective monitoring became essential to maintaining system reliability and performance. End-to-end observability across a complex, multi-stage pipeline proved invaluable for troubleshooting and continuous optimization. It let the team trace a single document’s journey from upload through text extraction, embedding generation, and vector storage.

CORTO built this observability in two complementary layers, each answering a different question.

Layer 1: Service-level observability with AWS Application Signals. CORTO instruments ECS tasks, AWS Lambda functions, and Aurora database connections with AWS Application Signals, which provides distributed tracing, request-level visibility, and error rate tracking across service boundaries. This answers the question “is each service healthy,” and it remains the foundation of the platform’s service-level monitoring today.

Layer 2: Document-level pipeline correlation. Because every document passes through four distinct stages, the team added a complementary layer of pipeline-specific instrumentation that correlates one document’s events across ingestion, extraction, embedding, and storage. This answers a different question: “where did this specific document stop.” Running both layers together gives the team service health and per-document lineage in the same view.

The combined monitoring approach delivered measurable operational improvements:

• Faster root cause identification across distributed system components.
• Reduced mean time to resolution (MTTR) from hours to minutes for production incidents.
• Better visibility into performance bottlenecks across the entire pipeline.
• Proactive issue detection before customer impact.

Current operational challenges

The Aurora PostgreSQL tier has been stable at production scale, and the challenges CORTO continues to work on sit in the ingestion pipeline and the AI infrastructure layers above it.

Root cause analysis: Despite improved observability, identifying the precise origin of issues in a distributed system remains difficult. Time to resolution remains higher than we want, and the path forward is AI-powered root cause analysis, which can surface the most likely failure points and provide one to two actionable steps in the right direction.

Real-time ingestion management: Prioritizing which documents need immediate embedding updates remains an active challenge. The current approach relies on queue-based prioritization with business logic, but the goal is more intelligent, predictive prioritization based on usage patterns that anticipates demand rather than responding to it.

Infrastructure management: Managing the AI cluster and associated services effectively at this scale requires ongoing attention. The operational goal is to reduce overhead through greater automation while maintaining the reliability that legal workflows demand.

Cost control: AI infrastructure costs require constant vigilance. Rather than treating optimization as a one-time architectural exercise, CORTO applies ongoing cost discipline across compute, database, text extraction, and model serving layers, continuously refining as usage patterns evolve.

Results and business impact

For the law firms on the platform, these gains compound: faster research cycles, lower per-matter costs, and the ability to surface relevant precedent from decades of accumulated knowledge in seconds rather than hours. CORTO is designed to give lawyers their time back. Pilot data signals up to 15 hours of recovered productive capacity per professional per week, time redirected from manual document review toward accelerated higher-value legal judgment, strategic counsel, and elevated client service.

The architectural decisions described above translate directly into measurable business outcomes. The following table summarizes key metrics across CORTO’s evolution from prototype to production scale:

Conclusion

CORTO’s journey demonstrates that purpose-built vector databases are not a prerequisite for production AI at scale. By building on existing PostgreSQL expertise, designing for multi-tenancy from the outset, and treating optimization as a continuous discipline, the platform evolved from rapid prototype development into a globally deployed production system.

The architectural decisions that made this possible emerged from real operational experience rather than upfront design. For teams building similar systems, the lesson is not to replicate CORTO’s architecture exactly, but to build with enough flexibility that yours can evolve as needs change. What you can learn from this approach: your existing PostgreSQL expertise transfers directly to vector search workloads. Embedding model selection should be driven by retrieval benchmarks on your domain data, not dimension count alone. Logical multi-tenant isolation simplifies your compliance story. And observability investment pays for itself at scale through faster incident resolution.

For the legal industry, the impact is transformative: decades of legal precedent now accessible in real time, giving legal teams back the hours once spent on manual research and turning that time into strategic counsel and better client outcomes. As the platform continues to scale, AI-powered operations and predictive ingestion will further extend this foundation.

To get started with pgvector on Aurora PostgreSQL, see the Amazon Aurora PostgreSQL pgvector documentation. For multi-tenant design patterns, refer to the Aurora PostgreSQL best practices guide.

About the authors