Nvidia Nemotron-3-Super-120B-FP8 Stained Glass Enabled

Organizations in many sectors (including, but not limited to healthcare, financial services, and legal) need to run large language models over confidential data - but shared GPU infrastructure and multi-tenant environments create exposure risk. Protopia AI's Stained Glass Transform (SGT, not included in this marketplace product) protects model inputs through stochastic transformation. This listing is a companion model endpoint configured to accept SGT-protected representations as well as plaintext, and return protected responses with Stained Glass Output Protection.

Protopia AI's Stained Glass Output Protection encrypts tokens as they are generated by an LLM. The generated response is never fully available in plaintext on the compute infrastructure. Raw data never leaves its trust zone-only these protected re-representations of the data reach the compute infrastructure.

Stained Glass Output Protection encrypts each token as it is generated by the LLM. Every request establishes a secure key exchange to individually encrypt each generated token. The plaintext response never exists in its entirety on the compute infrastructure. This prevents leakage via logs, dumps, state, etc. Only the authorized client holding its private key can decrypt the response.

Not included in this marketplace product, the companion Stained Glass Transform (SGT) transforms your input data into unintelligible prompt embeddings (prompt_embeds) before being sent to the compute infrastructure. The plaintext never leaves the data owner's trust zone. This [SGT-Enabled Nvidia Nemotron-3-Super-120B-FP8] supports SGT-transformed inputs, processing protected representations directly with no decoder on the endpoint.

This roundtrip protection pipeline means that neither the input nor the output is ever exposed on the inference host - a critical requirement for regulated industries handling patient records, financial documents, or privileged legal communications.

A financial services firm running retrieval-augmented generation (RAG) over proprietary research documents on shared cloud GPUs can deploy this listing to ensure that neither the retrieved context (input) nor the generated summaries (output) are visible to the infrastructure operator or co-tenants. The same pattern applies to healthcare organizations summarizing patient records or legal teams analyzing privileged communications.

The underlying NVIDIA Nemotron-3-Super-120B-A12B model was pre-trained on over 25 trillion tokens spanning code, math, science, and general knowledge across 20 languages, then instruction-tuned for tool calling, structured outputs, and long-context retrieval. NVIDIA reports benchmark results including 86.01 on MMLU and 90.67 on GSM8K (8-shot). This listing serves an FP8-quantized variant for reduced memory footprint and inference latency, which may show minor accuracy variation from published BF16 figures.

Served on vLLM with tensor parallelism that auto-sizes to the deployed instance's GPU count. Verified serving prompt_embeds correctly across all supported real-time instance sizes.

Note: despite the delivery configuration listing a recommended batch transform instance, batch (asynchronous) inference is not currently supported -- no AWS SageMaker batch transform instance type is capable of serving a model this large. Only real-time inference is supported today.

Unlike homomorphic encryption, SGT operates with low compute overhead on commodity hardware - no specialized TEE hardware required. Unlike differential privacy approaches that inject noise, SGT preserves full data fidelity so model accuracy is maintained. This frees powerful GPUs for inference rather than encryption overhead, and integrates into existing AI pipelines with minimal latency impact.

To evaluate this solution or schedule a technical demo with our solutions engineering team, contact us at contact@protopia.ai . We support deployments across on-premises, multi-tenant cloud, and edge environments.

Inputs can be passed as prompt_embeds (transformed embeddings). Outputs are returned as tokenwise-encrypted payloads using a secure key exchange. See the sample Jupyter notebook (https://github.com/protopia-ai/sagemaker-marketplace/blob/main/invoke_sgt_nemotron_endpoint.ipynb ) for a full walkthrough of payload structure, decryption steps, and SDK usage.