Quilt Connected Lab — Wet-Lab Data Management on AWS

Life sciences teams generate data faster than they can organize it. Every sequencer, flow cytometer, mass spectrometer, and plate reader produces files that end up on local drives, instrument computers, or ad-hoc S3 folders — invisible to the rest of the organization.

Connected Lab solves this by turning raw S3 objects into governed, searchable data packages.

HOW DATA MOVES

Quilt does not move data. Raw instrument files are transferred to Amazon S3 by your existing infrastructure — AWS Storage Gateway for on-premises instruments, AWS DataSync for batch transfers, direct S3 upload via CLI or SDK, or any pipeline that writes to S3. Quilt's role begins once data lands in the bucket.

HOW QUILT PROCESSES NEW DATA

When objects arrive in an S3 bucket managed by Quilt, the platform's event-driven indexing system (SNS/SQS notifications) detects the new files and indexes them in Elasticsearch — both shallow indexing (file name, size, metadata) and deep indexing of supported file contents (CSV, Parquet, PDF, Jupyter notebooks, FASTQ, and more). Objects become searchable in near-realtime.

Scientists or pipelines then create Quilt Packages — immutable, versioned bundles of related files with metadata — via the Python SDK, the Quilt web catalog, or automatically via the Nextflow plugin. Metadata Workflows (JSON Schema-based validation gates) enforce that every package includes required labels, controlled vocabularies, and documentation before it can be pushed to a bucket.

WHAT SCIENTISTS EXPERIENCE

Scientists open the Quilt web catalog, run a metadata or full-text search, and find the exact dataset they need in seconds. They can preview files inline (images, CSVs, notebooks, PDFs, FASTQ, BAM), browse package version history, and download or programmatically access data via the Python SDK. No S3 console navigation. No asking bioinformatics for file paths.

WHAT IT INCLUDES

• Event-driven indexing of S3 objects via SNS/SQS with near-realtime Elasticsearch updates
• Deep content indexing for CSV, Parquet, JSON, PDF, PPTX, Jupyter notebooks, and more
• Immutable, versioned Quilt Packages with cryptographic hash verification (SHA-256)
• Metadata Workflows: JSON Schema validation gates that enforce required metadata, controlled vocabularies, and README files before packages are accepted
• Web catalog with inline file preview, search, and package browsing
• Python SDK (quilt3) for programmatic package creation, access, and automation
• Nextflow plugin (nf-quilt) for automatic packaging of pipeline outputs
• Package promotion across buckets (raw → staging → production) with per-bucket workflow rules
• AWS IAM-enforced access control with role-based policies managed in the Quilt admin panel
• Full audit trail via AWS CloudTrail integration
• Deploys as a CloudFormation stack in your AWS account (ECS Fargate, Elasticsearch, RDS, Lambda)

PROVEN AT SCALE

Resilience (biomanufacturing): Replaced 3 legacy platforms. 200+ scientists onboarded in month one. NGS processing cut from 7–10 weeks to under 1 hour. 3,000+ hours saved per month. $3M cost savings. 7.6x ROI.

Tessera Therapeutics (gene writing): Over 1PB centralized. Nextflow plugin auto-packages every pipeline run. 3x faster NGS analysis, 50% reduction in data retrieval time, >80% daily usage across scientists and data engineers.

Entact Bio (precision medicine): 90%+ reduction in lookup time. 3x dataset reuse. Audit-ready for IND filing.

Inari Agriculture: 50% faster retrieval. Scientists self-serve; data requests to IT eliminated.