jina-embeddings-v2-base-code

Name: jina-embeddings-v2-base-code
Author: jinaai

by jinaai

Code embeddings over 30 programming languages, and the model behind Mixpeek's code index

161Mparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

jinaai/jina-embeddings-v2-base-code

Feature URI

Overview

Code is not prose, and embedding it with a text model loses the things that make code findable: identifier structure, call relationships, the difference between a docstring and the function it describes. This model was trained on the github-code corpus and then on more than 150 million pairs of coding questions, answers and docstring-to-source, which is why a natural-language question retrieves a function rather than a paragraph about the function.

Mixpeek runs it inside web_scraper@v1 as the code index. Crawl a documentation site and the prose lands in an E5 text index while the code blocks land here, so a question about how to call something reaches the snippet rather than the sentence next to it.

The context window is the other reason it is here: 8,192 tokens, extrapolated from a 512-token training length by ALiBi, which is enough for a whole file rather than a fragment.

Architecture

JinaBERT, a BERT variant using the symmetric bidirectional form of ALiBi in place of learned position embeddings, 160,869,120 parameters. ALiBi is what lets a model trained at 512 tokens extrapolate to 8,192 without retraining. The backbone jina-bert-v2-base-code is pretrained on github-code, then trained on Jina's collection of over 150 million coding question-answer and docstring-source pairs.

Mixpeek SDK Integration

import { Mixpeek } from "mixpeek";

const mx = new Mixpeek({ apiKey: "API_KEY" });

// Managed, and real: web_scraper@v1 runs this model as its code index
// (jinaai__jina_embeddings_v2_base_code, 768-d), beside an E5 text index for
// prose and a SigLIP index for images on the same crawl.
const collection = await mx.collections.create({
  namespace_id: "my-namespace",
  collection_name: "docs-site",
  source: { type: "bucket", bucket_ids: ["bkt_your_bucket"] },
  feature_extractor: {
    feature_extractor_name: "web_scraper",
    version: "v1",
  },
});

Capabilities

English plus 30 widely used programming languages in one space
8,192-token sequence length, enough for a file rather than a fragment
Natural-language question to code retrieval, not just code to code
161M parameters, small enough to run over a large repository

Use Cases on Mixpeek

The code index on a crawled documentation site or SDK reference

Answering a how-do-I question with the function that does it

Finding the implementation behind a docstring across repositories

Technical support search where the answer is a snippet

Performance

Input Size8,192 tokens via ALiBi extrapolation from a 512-token training length

Embedding Dim768 as indexed by Mixpeek

GPU LatencyInput dependent

GPU ThroughputBatch dependent

GPU MemoryModel dependent

Jina publishes no latency figures on the model card and Mixpeek has not published its own, so none are quoted here.

Common Pipeline Companions

intfloat/multilingual-e5-large-instruct

The prose index beside this code index on the same crawl

microsoft/codebert-base

Earlier code embedding baseline to compare against

BAAI/bge-reranker-v2-m3

Rescore code candidates before they are shown

Frequently Asked Questions

Which programming languages does it cover?

English plus 30 widely used programming languages, per Jina's model card. The backbone is pretrained on the github-code dataset, so coverage follows what is common there. A language outside that set will still embed, because nothing rejects it, and it will retrieve worse in a way nothing reports. Check recall on your own corpus if your stack is unusual.

Is 8,192 tokens real, or a claim about the architecture?

It is extrapolation, and Jina says so. The model was trained at 512 tokens and reaches 8,192 through ALiBi, which biases attention by distance rather than learning position embeddings, so longer sequences than were ever trained on remain coherent. It works, and it is not the same thing as having been trained at that length. If retrieval quality matters more than fitting a whole file in one vector, chunk anyway.

Does Mixpeek run this outside web_scraper?

No. GET /v1/discovery/extractors lists jinaai__jina_embeddings_v2_base_code only under web_scraper@v1, as the code index. A bucket of source files processed through text_extractor@v1 gets E5 embeddings, which are general-purpose text embeddings and weaker on code. To index a repository with this model specifically, either crawl it or run the model yourself and upsert.

Can I query it with a natural-language question rather than code?

Yes, that is what the 150 million question-answer and docstring-source pairs were for. The model was trained so a question and the code that answers it land near each other, which is the asymmetric case a code-to-code model does not handle. Querying with a code fragment also works, for finding similar implementations.

How does it compare to embedding code with a general text model?

Jina's card publishes no head-to-head table, so no number is quoted here. The mechanism is the part you can reason about: a general text model was trained on prose, where identifier names, punctuation density and structure carry very different weight than they do in source. The honest way to settle it for your corpus is to run both over a sample of real questions and compare recall at 10, which is a morning of work and beats either of us guessing.