ms-marco-MiniLM-L6-v2

Name: ms-marco-MiniLM-L6-v2
Author: cross-encoder

by cross-encoder

The small reranker behind many search stacks: 23M parameters, 512 tokens for query and passage together

339likes

23Mparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

cross-encoder/ms-marco-MiniLM-L6-v2

Feature URI

Deploy ms-marco-MiniLM-L6-v2

Single-tenant

Mixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.

Read the customer-hosted guide Talk to us about single-tenant

Overview

ms-marco-MiniLM-L6-v2 answers one question in a retrieval pipeline: given this query and this passage, how relevant is the passage? It reads the two together and returns a single score, so it runs after a vector search has produced a shortlist and reorders the top of that list. It is downloaded 87.9 million times a month, and its size is the reason: 23 million parameters is cheap enough to score a hundred candidates per query.

The limit to plan around is the shared window. The query and the passage are concatenated into one BERT input capped at 512 positions, so a long query eats into the passage budget, and anything past the cap is cut before scoring. Chunk passages so the relevant span sits near the start, because relevance that lives late in a long passage never reaches the model.

It was trained on the MS MARCO passage ranking task, which pairs English search queries with English web passages. Other languages, source code and noisy speech transcripts are outside that training data, and a multilingual cross-encoder is the better fit for them.

Architecture

MiniLM-style BERT encoder, 6 layers, 384-dimensional hidden state, 12 attention heads, 1536-dimensional feed-forward, 30,522-token vocabulary, loaded as BertForSequenceClassification with one output label. 22,713,601 float parameters; the safetensors total of 22,714,113 includes a 512-entry position-id buffer. max_position_embeddings is 512, and that budget covers the query and the passage together because they enter as one sequence pair.

Mixpeek SDK Integration

import { Mixpeek } from "mixpeek";

const mx = new Mixpeek({ apiKey: "API_KEY" });

// A reranker is a retriever STAGE, because it needs the query, and the query
// exists only at search time. The rerank stage's built-in service is
// BAAI/bge-reranker-v2-m3. Running ms-marco-MiniLM-L6-v2 in that stage means
// registering it as a custom reranker plugin and referencing it through
// feature_uri, which is an Enterprise path. Contract read from
// GET /v1/discovery/stages.
const retriever = await mx.retrievers.create({
  namespace_id: "my-namespace",
  retriever_name: "search-then-rerank",
  stages: [
    {
      stage_name: "candidates",
      stage_id: "feature_search",
      parameters: { limit: 100 },
    },
    {
      stage_name: "rerank_results",
      stage_id: "rerank",
      parameters: {
        inference_name: "BAAI__bge_reranker_v2_m3",
        query: "{{INPUT.query}}",
        document_field: "content",
        top_k: 10,
      },
    },
  ],
});

Capabilities

One relevance score per query and passage pair, computed by reading both together
Card-reported 1,800 passages per second on a V100 GPU
Part of a family of MS MARCO cross-encoders with one interface (L2, L4, L6, L12), so speed against accuracy is a checkpoint swap
Apache-2.0 license

Use Cases on Mixpeek

Reordering the top 100 results of a vector search down to the 10 a user sees

Scoring transcript or caption chunks before they go into a RAG prompt

An English baseline to measure a larger reranker against

Dropping low-relevance passages with a score threshold tuned on your own queries

Benchmarks

Dataset	Metric	Score	Source
TREC Deep Learning 2019	NDCG@10	74.30	Model card: cross-encoder/ms-marco-MiniLM-L6-v2
MS MARCO Passage (dev)	MRR@10	39.01	Model card: cross-encoder/ms-marco-MiniLM-L6-v2

Performance

Input Size512 tokens for query and passage combined

GPU LatencyInput dependent

GPU ThroughputBatch dependent

GPU MemoryModel dependent

The model card reports 1,800 passages per second on a V100 GPU, with the L12 variant at 960 and the L4 variant at 2,500. We have not measured it on our own hardware.

Common Pipeline Companions

BAAI/bge-reranker-v2-m3

The multilingual reranker Mixpeek's rerank stage runs by default

sentence-transformers/all-MiniLM-L6-v2

A small English first-stage embedding model to produce the shortlist this model reorders

Frequently Asked Questions

Why does ms-marco-MiniLM-L6-v2 score every candidate separately?

It is a cross-encoder: the query and the passage enter the network as one sequence and attention runs across both. That is where its accuracy comes from, and it also means nothing can be computed ahead of time at ingest. Cost grows with the number of candidates per query, so it belongs after a first stage that has already cut the corpus to a shortlist.

What does the 512-token limit cover?

The query and the passage together. BERT's max_position_embeddings is 512 and the pair is concatenated before encoding, so a 40-token query leaves 469 tokens for the passage after the three special tokens. Text past the cap is truncated before scoring.

Should I use the L6 or the L12 version?

The card's own table shows L12-v2 at 74.31 NDCG@10 on TREC DL 2019 against 74.30 for L6-v2, at 960 passages per second against 1,800. On those numbers L6 gives up almost no accuracy for close to twice the throughput. Check both on your own queries before relying on either figure.

Can Mixpeek's rerank stage run this model?

Only as a custom reranker. inference_name selects reranking services Mixpeek hosts, and the default is BAAI/bge-reranker-v2-m3. This model would be registered as a custom reranker plugin and referenced through feature_uri, which is an Enterprise capability.