multilingual-e5-large

Name: multilingual-e5-large
Author: intfloat

by intfloat

Microsoft's 560M multilingual text embedder: 1,024-dimension vectors for about 100 languages, with "query: " and "passage: " prefixes for retrieval

1,281likes

560Mparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

intfloat/multilingual-e5-large

Feature URI

Deploy multilingual-e5-large

Single-tenant

Mixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.

Read the customer-hosted guide Talk to us about single-tenant

Overview

multilingual-e5-large is a text embedding model from Microsoft researchers, published on Hugging Face in June 2023 under the MIT license and described in the 2024 Multilingual E5 technical report. It starts from XLM-RoBERTa-large (24 layers, about 560M parameters) and turns text of up to 512 tokens in about 100 languages into a 1,024-dimension vector. Queries take the prefix "query: " and documents take "passage: ", in every language.

On the Mr. TyDi retrieval benchmark across 11 languages the card reports 70.5 MRR@10, against 33.3 for BM25 and 65.9 for the smaller multilingual-e5-base. It is one of the most downloaded embedding models on Hugging Face, at about 6.5 million downloads a month.

Architecture

An XLM-RoBERTa-large encoder (24 layers, hidden size 1,024, 512-token maximum input) with average pooling over the last hidden states, followed by L2 normalization. Trained in two stages: contrastive pre-training on about 5 billion weakly supervised multilingual text pairs (mC4, CC News, NLLB translation pairs, Wikipedia, Reddit and others), then supervised fine-tuning on labeled retrieval sets including MS MARCO, NQ, Mr. TyDi and MIRACL.

Mixpeek SDK Integration

# Embed documents with multilingual-e5-large. Documents take the "passage: " prefix.
# pip install -U sentence-transformers
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("intfloat/multilingual-e5-large")
docs = ["passage: El gato duerme en el sofá.", "passage: The train leaves at nine."]
doc_vecs = model.encode(docs, normalize_embeddings=True)   # 1,024 dims

Capabilities

1,024-dimension text embeddings in about 100 languages
One vector space across languages, so a query in one language can match a passage in another
Inputs up to 512 tokens
Loads with Sentence Transformers and Transformers
MIT license

Use Cases on Mixpeek

Retrieval for RAG over documents, transcripts and captions in many languages

Cross-language search, such as English queries over Spanish or Japanese content

Semantic deduplication and clustering of multilingual text

A self-hosted multilingual baseline to compare against paid embedding APIs

Benchmarks

Dataset	Metric	Score	Source
Mr. TyDi, 11 languages	Average MRR@10	70.5	intfloat model card
Mr. TyDi, English	MRR@10	60.8	intfloat model card
Mr. TyDi, Japanese	MRR@10	62.5	intfloat model card

Performance

Input SizeText up to 512 tokens; about 100 languages

GPU LatencyInput dependent

GPU ThroughputBatch dependent

GPU MemoryModel dependent

Longer passages are truncated at 512 tokens, so chunk documents before embedding. The card says leaving out the "query: " and "passage: " prefixes lowers quality, and that low-resource languages may score lower. The Mr. TyDi figures are from the card; we have not measured it.

Common Pipeline Companions

BAAI/bge-reranker-v2-m3

A multilingual reranker for the top results

intfloat/multilingual-e5-large-instruct

The instruction-tuned version, which takes a task description with each query

BAAI/bge-m3

A multilingual alternative with 8,192-token inputs and sparse vectors

Frequently Asked Questions

What is multilingual-e5-large?

An open multilingual text embedding model from Microsoft researchers with about 560M parameters, released under the MIT license. It produces 1,024-dimension vectors from inputs of up to 512 tokens in about 100 languages and reports 70.5 average MRR@10 on the Mr. TyDi retrieval benchmark.

Do I need the "query: " and "passage: " prefixes?

Yes. The model was trained with them, and the card says quality drops without them. Prefix search queries with "query: " and indexed documents with "passage: ", in English as in every other language. For tasks other than retrieval, such as clustering, use "query: " for every text.

Can multilingual-e5-large search across languages?

Yes. All languages share one vector space, so a query in English can retrieve a passage written in Spanish or Japanese. Test it on your own language pairs, since the card notes that low-resource languages may score lower.

What is the difference between multilingual-e5-large and multilingual-e5-large-instruct?

Both share the XLM-RoBERTa-large base and 1,024-dimension output. The instruct version takes a one-sentence task description in front of each query in place of the "query: " prefix, which lets one model serve retrieval, classification and clustering with different instructions.

How do I use multilingual-e5-large with Mixpeek?

Mixpeek's managed extractors do not include multilingual-e5-large today; running it as an extractor means registering a custom model, an Enterprise path. To compare it with your current search first, embed a sample of your documents as in the example on this page.