LaBSE

by sentence-transformers

Language-agnostic sentence embeddings that put 109 languages in one shared space

755Kdl/month

471Mparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

sentence-transformers/LaBSE

Feature URI

mixpeek://text_extractor@v1/labse_v1

Overview

LaBSE (Language-agnostic BERT Sentence Embedding) is trained so that a sentence and its translation land in nearly the same place in vector space, across 109 languages. That property is the whole point: it means a query in one language retrieves matching content in every other language with no translation step and no language detection anywhere in the request path.

On Mixpeek it is the text side of a cross-lingual archive. Index documents and transcripts in whatever language they arrive in, and a single index answers queries in any of the 109, which is what makes a mixed-language library searchable without routing logic that has to guess a document's language correctly.

Architecture

Dual-encoder BERT trained with a translation-ranking objective over parallel sentence pairs, producing a 768-dimension sentence embedding. Distributed for sentence-transformers with PyTorch, TensorFlow, JAX and ONNX weights.

Mixpeek SDK Integration

import { Mixpeek } from "mixpeek";

const mixpeek = new Mixpeek({ apiKey: process.env.MIXPEEK_API_KEY });

// One index, every language. No language detection on the write path.
await mixpeek.collections.create({
  collection_name: "multilingual_docs",
  feature_extractors: [
    { extractor: "mixpeek://text_extractor@v1/labse_v1" },
  ],
});