NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Assortment Whose 8B Checkpoint Ranks #1 on RTEB

Embedding fashions determine which passages an agent ever sees. NVIDIA launched Nemotron 3 Embed mannequin to work on that layer. It targets production-scale RAG, agentic retrieval, code retrieval, and agent reminiscence.

What’s Nemotron 3 Embed?

The mannequin assortment consists of three open checkpoints. Nemotron-3-Embed-8B-BF16 is the accuracy-first choice. Nemotron-3-Embed-1B-BF16 carries the identical design right into a smaller footprint. Nemotron-3-Embed-1B-NVFP4 is the Blackwell-optimized 4-bit path.

All three are transformer encoders educated with bidirectional consideration masking. The ultimate embedding comes from common pooling over token-level representations. Most sequence size is 32,768 tokens on each checkpoint.

Every mannequin was evaluated throughout 34 languages. All three carry the OpenMDW License Settlement, model 1.1 (OpenMDW-1.1). Notably, the bases are Mistral fashions. The 8B is constructed with Ministral-3-8B-Instruct-2512. Each 1B variants use Ministral-3-3B-Instruct-2512.

Efficiency

Nemotron-3-Embed-8B-BF16 ranks #1 general on RTEB (as of July 17 2026), the Retrieval Embedding Benchmark. Analysis covers its 16 public duties. Each determine beneath is common NDCG@10, at mannequin sequence size 4096.

Two gaps are value noting. The 1B positive aspects 10.4 RTEB factors over llama-nemotron-embed-vl-1b-v2, the prior-generation baseline. Individually, NVFP4 prices 0.38 RTEB factors towards its BF16 father or mother, or 99.5% retention.

How the 1B Mannequin was Constructed?

These 1B scores come from a compression pipeline, not a smaller coaching run. The father or mother was nemotron-3-embed-3b, pruned and distilled throughout two iterative rounds.

First, the 3B father or mother was pruned to 2B utilizing NVIDIA ModelOpt mcore_minitron Neural Structure Search (NAS). The search covers hidden width, FFN dimension, consideration heads, and depth. It then picks one of the best candidate from the top-10 Pareto entrance. A 50k in-domain calibration corpus scored these candidates.

Subsequent, the 2B mannequin was distilled from the fine-tuned 8B embedding instructor. Distillation mixed cosine distance loss (COS) and imply squared error (MSE) loss. The information mix was multilingual and in-domain. Lastly, the identical process repeated to supply the 1.14B checkpoint.

The NVFP4 Serving Tradeoff

Compression then continues into the serving format. Quantization hit weights and activations of linear layers solely, focusing on the NVFP4 knowledge kind. The analysis group used nvidia-modelopt v0.45.0. Quantization-Conscious Distillation (QAD) adopted, primarily to recuperate accuracy on lengthy inputs.

Calibration used 512 samples: 256 queries and 256 passages from abisee/cnn_dailymail. QAD coaching used 20k samples.

The rsesearch group experiences NVFP4 on Blackwell delivers as much as 2x larger throughput than BF16. It retains 99%+ of BF16 retrieval accuracy. The NVFP4 card additionally paperwork dynamic embedding sizes. You may slice the 2048-d vector from the begin to 1024 or 512 dimensions. Re-normalize afterward.

Interactive Explainer: The 5-Stage Retrieval Path

Earlier than touching code, watch the trail run. It animates prefixing, bidirectional encoding, common pooling, L2 normalization, and dot-product scoring. Scores come from every card’s printed anticipated output.

Deployment Matrix

As that walkthrough implies, the checkpoints don’t share runtime paths.

Alongside the checkpoints, NVIDIA analysis group launched an optimized NIM microservice for the 1B mannequin. The Rust-based NIM matches or outperforms the vLLM checkpoint on GB200 and RTX PRO 6000. NVIDIA examined enter sequence lengths of 256 and 1024. Individually, NVIDIA NeMo AutoModel recipes cowl fine-tuning and distillation.

Utilizing It in Code

With these paths in thoughts, prefixes come first. Queries take question: and paperwork take passage: . Embeddings are L2-normalized, so dot product equals cosine similarity.

# pip set up --upgrade "transformers>=5.2.0" "sentence-transformers>=5.4.1"
import torch
from sentence_transformers import SentenceTransformer

QUERIES = ["How can someone reduce exposure to pollen during allergy season?"]
DOCUMENTS = ["People with pollen allergy can reduce exposure by staying indoors "
             "on dry, windy days, avoiding early-morning outdoor activity, and "
             "going outside after rain when pollen levels are lower."]

mannequin = SentenceTransformer(
    "nvidia/Nemotron-3-Embed-8B-BF16",
    system="cuda",
    model_kwargs={"dtype": torch.bfloat16,
                  # use "sdpa" if FlashAttention-2 is unavailable
                  "attn_implementation": "flash_attention_2"},
    processor_kwargs={"padding_side": "left"},
)
mannequin.max_seq_length = 32768

q = mannequin.encode_query(QUERIES, batch_size=1, convert_to_tensor=True)
d = mannequin.encode_document(DOCUMENTS, batch_size=1, convert_to_tensor=True)
print(mannequin.similarity(q, d))  # card's printed q[3]/d[3] rating: 0.8008

encode_query and encode_document learn the saved prompts. So that you by no means add prefixes by hand. For serving, /v2/embed applies them from input_type as a substitute:

vllm serve nvidia/Nemotron-3-Embed-1B-NVFP4 
  --max-model-len 4096 
  --max-num-batched-tokens 4096 
  --max-cudagraph-capture-size 4096

import numpy as np, requests

def embed(input_type: str, texts: record[str]) -> np.ndarray:
    r = requests.publish(
        "http://localhost:8000/v2/embed",
        json={"mannequin": "nvidia/Nemotron-3-Embed-1B-NVFP4",
              "input_type": input_type,          # "question" or "doc"
              "texts": texts,
              "embedding_types": ["float"],
              "truncate": "END"},
        timeout=120,
    )
    r.raise_for_status()
    return np.array(r.json()["embeddings"]["float"], dtype=np.float32)

scores = embed("question", QUERIES) @ embed("doc", DOCUMENTS).T

Use Instances With Examples

• Multilingual enterprise search: A help group indexes Hindi, Japanese, and English tickets collectively. As a result of retrieval is cross-lingual, a German question can floor a Japanese decision notice.
• Code retrieval: Coaching included coir_apps, coir_cosqa, synthetic_text2sql, and SWE-bench. Pure-language-to-code lookup is due to this fact nearer to in-distribution.
• Agent reminiscence: The 32,768-token restrict lets an agent embed lengthy dialog summaries with out aggressive chunking.
• Price-tiered RAG: Serve 1B-NVFP4 for high-volume recall, and route onerous queries to the 8B. As a result of widths differ, this wants two indexes.

Key Takeaways

• Nemotron-3-Embed-8B-BF16 ranks #1 on RTEB at 78.46 avg NDCG@10.
• Three open checkpoints span 8B BF16, 1B BF16, and 1B NVFP4.
• NVFP4 retains 99%+ of BF16 accuracy at as much as 2x Blackwell throughput.
• The 1B got here from ModelOpt NAS pruning plus COS+MSE distillation from the 8B.
• All checkpoints use OpenMDW-1.1 and help 32,768-token inputs.

Try the NVIDIA launch publish on Hugging Face, Nemotron 3 Embed assortment, 8B-BF16 card, 1B-BF16 card and 1B-NVFP4 card. Additionally, be happy to observe us on Twitter and don’t overlook to affix our 150k+ML SubReddit and Subscribe to our E-newsletter. Wait! are you on telegram? now you’ll be able to be part of us on telegram as effectively.

Must associate with us for selling your GitHub Repo OR Hugging Face Web page OR Product Launch OR Webinar and so forth.? Join with us

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is dedicated to harnessing the potential of Synthetic Intelligence for social good. His most up-to-date endeavor is the launch of an Synthetic Intelligence Media Platform, Marktechpost, which stands out for its in-depth protection of machine studying and deep studying information that’s each technically sound and simply comprehensible by a large viewers. The platform boasts of over 2 million month-to-month views, illustrating its recognition amongst audiences.