Served on MixpeekEmbeddingVerified 2026-11-05
ColBERTv2
colbert-ir/colbertv2.0
ColBERTv2 is a text embedding model (MIT). Served on Mixpeek, scaling to zero when idle.
A Ray Serve app in Mixpeek's inference registry hosts it. It starts at zero replicas and wakes on first use.
New opens the Designer in Studio. For an existing namespace, open it in Studio and use its Designer tab.
No demo for ColBERTv2 yet
A video, image, slideshow or playground goes here once we record one on our own data.
What it is good and bad at
- Best at
- Late-interaction text retrieval and reranking (MaxSim over one 128-d vector per token); strong on out-of-domain and term-heavy queries.
- Worst at
- Storage and cold reads: about 36 KB a 275-token document; English-centric BERT-base vocabulary; text only.
- Task
- Embedding
- Input
- Text
- Output
- Multi-vector
- Use cases
- Text search, Reranking
Templates that use it
No template uses this model yet. Browse templates.
Input and output
Input
- text
- max_tokens
- 512
- query_marker
- [unused0] as [Q]
- document_marker
- [unused1] as [D]
- query_padding
- [MASK] to 32 positions
Output
- vector
- dims
- 128
- distance
- cosine
- type
- multi_dense
How Mixpeek wires it
- Extractors
colbert_extractor@v1fixed
- Feature URIs
mixpeek://colbert_extractor@v1/colbertv2_v1
- Vector indexes
colbert_extractor_v1_tokens- Retriever stages
- maxsim_rerank, feature_search
- Combines with
- feature_search (E5) -> maxsim_rerank on mixpeek://colbert_extractor@v1/colbertv2_v1
How it runs
- Status
- Served on Mixpeek Ray Serve app in the inference registry; colbert_extractor batch uses ColBERTv2Batch. The embedding registry marks it PREVIEW.
- Serve apps
colbert_ir__colbertv2- Runtime
- transformers BertModel (add_pooling_layer=False) + AutoTokenizer, 768->128 linear head (ColBERTv2Encoder, engine code)
- Weights
- safetensors, no remote code
- Parameters
- 110 M
- Size on disk
- 438 MB
- Preprocessing
- Documents: [CLS] [D] tokens [SEP], truncated to 512. Queries: [CLS] [Q] tokens [SEP], padded with [MASK] to 32 positions.
- Postprocessing
- Project each token 768 -> 128 with the checkpoint linear head and L2-normalize; documents drop punctuation and padding tokens.
- Serving
- both, batch 16
- Hardware
- Runs on CPU (slower without a GPU)
Scale to zero
- Minimum replicas
- 0
- Maximum replicas
- 2
- Idle timeout
- 300 s
- Memory
- 2 GB
- CPUs
- 1
- GPUs
- 0
- Cold start, node warm
- Not recorded
- Cold start, new node
- Not recorded
Measured cost
Unmeasured
No committed throughput, latency, memory and cost measurement of the Serve app.
Our evaluations
| Dataset and metric | Score | Baseline | Date |
|---|---|---|---|
BEIR SciFact (5,183 documents) nDCG@10, exact MaxSim, engine encoder fp16 on MPS (k=10) research/goal-specs/mvs-scale-from-zero/RESULTS-4.36-1M-RERUN.md | 0.7063 | None | 2026-10-14 |
BEIR SciFact padded with 994,817 PubMed abstracts (1M documents) nDCG@10, exact MaxSim (k=10) research/goal-specs/mvs-scale-from-zero/RESULTS-4.36-1M-RERUN.md | 0.4414 | None | 2026-10-14 |
Questions
- What is ColBERTv2 best at?
- Late-interaction text retrieval and reranking (MaxSim over one 128-d vector per token); strong on out-of-domain and term-heavy queries.
- What are the limits of ColBERTv2?
- Storage and cold reads: about 36 KB a 275-token document; English-centric BERT-base vocabulary; text only.
- Can ColBERTv2 be used commercially?
- Yes. MIT on the Hugging Face repo; no use restrictions.
- How does ColBERTv2 run on Mixpeek?
- Ray Serve app in the inference registry; colbert_extractor batch uses ColBERTv2Batch. The embedding registry marks it PREVIEW.