S4 Weights - Lossless GPU Checkpoint Compression for PyTorch Training

S4 Weights is a transparent, lossless compression codec for the checkpoints a PyTorch training run writes - the model weights and the optimizer state. It is delivered as a GPU AMI you launch on your own Amazon EC2 instances and run inside your own VPC; your checkpoints are written to your own Amazon S3 bucket and never leave your account.

How it works: each tensor is split into byte planes (sign / exponent / mantissa) on the GPU and each plane is routed to the codec that suits it - the exponent plane through ANS entropy coding, the mantissa through GDeflate, the sign plane bit-packed (built on NVIDIA nvCOMP). For runs that checkpoint frequently, S4 Weights also stores the byte-XOR delta between consecutive checkpoints and compresses that, which is much smaller when most weights barely move between saves.

Lossless is the trust core: restore is byte-for-byte identical, verified against adversarial bit patterns (NaN, +/-Inf, denormal, -0.0) on every supported dtype. The AMI build itself fails if a compress -> decompress round-trip is not bit-exact on the build GPU, so a broken plane reassembly never reaches a customer image.

What you save depends on the data: all-bf16 / low-precision-optimizer checkpoints compress well (and better at scale); fp32-heavy checkpoints saved far apart compress little, so the product is honest about where it helps (see the documentation). Compression is always lossless and never expands a blob beyond a small fixed header.

Billing: a paid hourly + annual per-instance-type AMI. AWS meters the running instance-hours automatically; the runner calls RegisterUsage once at boot as a fail-closed entitlement check (an unentitled instance refuses to start) and does not meter per-usage by default. The runner runs as a non-root service and persists compressed checkpoints to the S3 registry bucket you configure. Support: aws-support@abyo.net .