gte-multilingual-base
by Alibaba-NLP
8192 tokens, 70+ languages, 768 dimensions, and 305M parameters
Alibaba-NLP/gte-multilingual-baseDeploy gte-multilingual-base
Single-tenantMixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.
Overview
gte-multilingual-base covers the case the sentence-transformers classics do not: long multilingual passages. Its card claims support for over 70 languages and a context window of 8192 tokens, at 305 million parameters, which is an order of magnitude smaller than the 7B instruct-style embedding models and cheap enough to run over a large corpus.
It pools from the CLS token rather than by mean, which matters if you are writing the encoding step yourself: mean pooling over this model's outputs produces a vector that works well enough to pass a smoke test and measurably worse than the intended one. The repo also ships sparse vector support and elastic dense output, so one model can produce a shorter vector when storage is the constraint.
It requires trust_remote_code, because the architecture is a custom one registered as model_type "new" rather than a standard transformers class. That is an operational fact worth surfacing before a security review finds it.
Architecture
Custom encoder registered as model_type "new", 12 layers, 768-dimensional hidden state, 305,369,089 parameters, 250,048-token vocabulary, max_position_embeddings 8192. CLS pooling (pooling_mode_cls_token true, mean pooling false), and sentence_bert_config.json sets max_seq_length to the full 8192. Loading it through transformers or sentence-transformers requires trust_remote_code.
Mixpeek SDK Integration
// Runs on your side; upsert the vectors. Two details decide whether the
// embeddings are any good: CLS pooling (not mean), and trust_remote_code=True
// when you load it.
const res = await fetch(
"https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
{
method: "POST",
headers: {
Authorization: "Bearer API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({
collection_id: "col_your_collection",
documents: [
{
document_id: "doc-2026-0412-section-2",
vectors: { "text-embedding": yourVector },
payload: { language: "ja", source_key: "contracts/2026/0412.pdf" },
},
],
}),
},
);Capabilities
- 8192-token context, so a long document section embeds without chunking
- Over 70 languages, per the model card, including cross-lingual retrieval
- 768-dimensional dense output with elastic shorter variants for storage savings
- Sparse term-weight vectors alongside the dense ones, for hybrid retrieval from one model
Use Cases on Mixpeek
Performance
Not benchmarked here. The card publishes MIRACL, MLDR, MKQA, BEIR and LoCo results, and the MTEB leaderboard carries independent scores.
Common Pipeline Companions
Frequently Asked Questions
Does gte-multilingual-base really handle 8192 tokens?
Its config sets max_position_embeddings to 8192 and the sentence-transformers config sets max_seq_length to the same, so nothing truncates below that. Whether a single 8192-token vector is useful for retrieval is a separate question: a vector averaged over that much text loses the specificity that makes a match precise, so long-window support is best used to avoid splitting a coherent section, not to index whole documents as one point.
Why does it need trust_remote_code?
The architecture is registered as model_type "new" with an auto_map pointing at code in the repo, so loading it executes Python shipped by the model author. Pin a revision and review that code if your environment has a policy about it.
CLS pooling or mean pooling?
CLS. The repo's 1_Pooling/config.json sets pooling_mode_cls_token true and pooling_mode_mean_tokens false. Mean pooling over this model produces vectors that retrieve plausibly and score worse, which is a failure that passes every smoke test.
Specification
Research Paper
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval
arxiv.orgBuild a pipeline with gte-multilingual-base
Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.
Run it on your own data, free