Llama-3.2-NV-EmbedQA-1B-v2

Multilingual text question-answering retrieval, transforming textual information into dense vector representations.

Overview

The NVIDIA NeMo Retriever Llama3.2 embedding model is optimized for multilingual and cross-lingual text question-answering retrieval with support for long documents (up to 8192 tokens) and dynamic embedding size (Matryoshka Embeddings). This model was evaluated on 26 languages: English, Arabic, Bengali, Chinese, Czech, Danish, Dutch, Finnish, French, German, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Norwegian, Persian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, and Turkish.

In addition to enabling multilingual and cross-lingual question-answering retrieval, this model reduces the data storage footprint by 35x through dynamic embedding sizing and support for longer token length, making it feasible to handle large-scale datasets efficiently.

For additional information please contact NVIDIA: https://www.nvidia.com/en-us/data-center/lp/aws-marketplace-offer

This model is ready for commercial use.

Highlights

The NeMo Retriever Llama3.2 embedding model is most suitable for users who want to build a multilingual question-and-answer application over a large text corpus, leveraging the latest dense retrieval technologies.
NVIDIA NIM, a part of the [NVIDIA AI Enterprise](https://www.nvidia.com/en-us/data-center/products/ai-enterprise/) software platform available on the [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-ozgjkov6vq3l6), is a set of easy-to-use microservices designed for secure, reliable deployment of high performance AI model inferencing.

Details

Sold by

NVIDIA

Unlock automation with AI agent solutions

Fast-track AI initiatives with agents, tools, and solutions from AWS Partners.

Explore AI agent solutions

Features and programs

Financing for AWS Marketplace purchases

AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.

View financing details

Pricing

Llama-3.2-NV-EmbedQA-1B-v2

Info

View purchase options

Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time.

Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.

Usage costs (35)

Info

Dimension	Description	Cost/host/hour
ml.g5.2xlarge Inference (Batch) Recommended	Model inference on the ml.g5.2xlarge instance type, batch mode	$1.00
ml.g5.2xlarge Inference (Real-Time) Recommended	Model inference on the ml.g5.2xlarge instance type, real-time mode	$1.00
ml.g5.xlarge Inference (Batch)	Model inference on the ml.g5.xlarge instance type, batch mode	$1.00
ml.g5.12xlarge Inference (Batch)	Model inference on the ml.g5.12xlarge instance type, batch mode	$4.00
ml.g5.8xlarge Inference (Batch)	Model inference on the ml.g5.8xlarge instance type, batch mode	$1.00
ml.g5.4xlarge Inference (Batch)	Model inference on the ml.g5.4xlarge instance type, batch mode	$1.00
ml.g5.48xlarge Inference (Batch)	Model inference on the ml.g5.48xlarge instance type, batch mode	$8.00
ml.g5.16xlarge Inference (Batch)	Model inference on the ml.g5.16xlarge instance type, batch mode	$1.00
ml.g5.24xlarge Inference (Batch)	Model inference on the ml.g5.24xlarge instance type, batch mode	$4.00
ml.g6.16xlarge Inference (Real-Time)	Model inference on the ml.g6.16xlarge instance type, real-time mode	$1.00

Vendor refund policy

No refund

How can we make this page better?

We'd like to hear your feedback and ideas on how to improve this page.

Legal

Vendor terms and conditions

Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .

Content disclaimer

Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.

Usage information

Info

Delivery details

Amazon SageMaker model

An Amazon SageMaker model package is a pre-trained machine learning model ready to use without additional training. Use the model package to create a model on Amazon SageMaker for real-time inference or batch processing. Amazon SageMaker is a fully managed platform for building, training, and deploying machine learning models at scale.

Deploy the model on Amazon SageMaker AI using the following options:

Real-time inference

Deploy the model as an API endpoint for your applications. When you send data to the endpoint, SageMaker processes it and returns results by API response. The endpoint runs continuously until you delete it. You're billed for software and SageMaker infrastructure costs while the endpoint runs. AWS Marketplace models don't support Amazon SageMaker Asynchronous Inference. For more information, see Deploy models for real-time inference .

Batch transform

Deploy the model to process batches of data stored in Amazon Simple Storage Service (Amazon S3). SageMaker runs the job, processes your data, and returns results to Amazon S3. When complete, SageMaker stops the model. You're billed for software and SageMaker infrastructure costs only during the batch job. Duration depends on your model, instance type, and dataset size. AWS Marketplace models don't support Amazon SageMaker Asynchronous Inference. For more information, see Batch transform for inference with Amazon SageMaker AI .

Version release notes

Added the NIM_SERVED_MODEL_NAME environment variable. Updated the LangChain Playbook to use the Llama-3.2-NV-EmbedQA-1B-v2 NIM.

Additional details

Inputs

Summary: The model accepts JSON requests that specifies the input text to be embedded.

{ "input": ["Hello world"], "model": "nvidia/llama-3.2-nv-embedqa-1b-v2", "input_type": "query" }'

Input MIME type: application/json

Real-time inference sample input data

https://docs.api.nvidia.com/nim/reference/nvidia-llama-3_2-nv-embedqa-1b-v2-infer

Batch transform sample input data

https://docs.api.nvidia.com/nim/reference/nvidia-llama-3_2-nv-embedqa-1b-v2-infer

Input data descriptions

The following table describes supported input data fields for real-time inference and batch transform.

Field name	Description	Constraints	Required
input	Input text to embed. Max length is 512 tokens.	Default value: "" Type: FreeText	No
model	ID of the embedding model.	Type: FreeText	Yes
input_type	passage is used when generating embeddings during indexing. query is used when generating embeddings during querying.	Default value: passage Type: Categorical Allowed values: passage, query	No
encoding_format	The format to return the embeddings in.	Default value: float Type: Categorical Allowed values: float, base64	No
truncate	Specifies how inputs longer than the maximum token length of the model are handled.	Default value: none Type: Categorical Allowed values: NONE, START, END	No

Resources

Vendor resources

Experience this NIM as hosted endpoint

Documentation

Support

Vendor support

Free support via NVIDIA NIM Developer Forum: https://forums.developer.nvidia.com/c/ai-data-science/nvidia-nim/

Global enterprise support with NVIDIA AI Enterprise subscription:

Get support

AWS infrastructure support

AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.

Get support

Customer reviews

Leave a review

Ratings and reviews

Info

0 ratings

5 star

4 star

3 star

2 star

1 star

0 AWS reviews

No customer reviews yet

Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.