[Models](/models)/[Text Extraction](/models?category=text_extraction)/microsoft/codebert-base

HFCode ExtractionMIT

# codebert-base

by microsoft

Pre-trained model for code understanding and generation

221Kdl/month

287likes

125Mparams

[HuggingFace](https://huggingface.co/microsoft/codebert-base)[Run it on your own data, free](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=model_page&utm_campaign=mvs_onramp)

Identifiers

Model ID

`microsoft/codebert-base`

## Deploy codebert-base

Single-tenant

Mixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.

[Read the customer-hosted guide](https://mixpeek.com/docs/customer-hosted/overview) [Talk to us about single-tenant](/demo)

## Overview

CodeBERT is a bimodal pre-trained model for programming languages and natural language. It supports code search, code documentation generation, and code-to-code translation across 6 programming languages.

On Mixpeek, CodeBERT extracts and embeds code blocks from documents, enabling semantic search over code content, find code snippets by describing what they do in natural language.

## Architecture

RoBERTa-base architecture (12 layers, 768-dim hidden, 12 attention heads) pre-trained on CodeSearchNet dataset with Masked Language Modeling and Replaced Token Detection on both NL and PL.

## Mixpeek SDK Integration

IngestRetrieve

// No extractor parameter takes a Hugging Face model id (checked against
// GET /v1/discovery/extractors, which returns 13), so codebert-base runs
// on your side and the output is upserted through POST
// /v1/namespaces/{namespace\_id}/documents/upsert. On Enterprise the other
// path is to upload the weights instead: POST /v1/namespaces/{id}/models
// accepts the huggingface format and a custom plugin loads them.
const res = await fetch(
  "https://api.mixpeek.com/v1/namespaces/ns\_your\_namespace/documents/upsert",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer API\_KEY",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      collection\_id: "col\_your\_collection",
      documents: \[
        {
          document\_id: "asset-00412",
          // The model produces text, so it lands in payload. Give the
          // collection a text vector index and embed that text to make it
          // searchable rather than only filterable.
          payload: { extracted\_text: modelOutput, source\_key: "archive/2026/asset-00412" },
          vectors: { "text-embedding": embeddingOfModelOutput },
        },
      \],
    }),
  },
);

// Managed alternative, if this exact model is not the requirement:
// universal\_extractor@v1 runs google/gemini-embedding-2
// (3072-d) over a bucket, with no inference of your own.

## Capabilities

*   Natural language code search
*   Code documentation generation
*   6 programming languages: Python, Java, JS, PHP, Ruby, Go
*   768-dimensional code embeddings

## Use Cases on Mixpeek

Search code repositories by natural language description

Extract and index code blocks from technical documentation

Code similarity detection for plagiarism or deduplication

## Benchmarks

Dataset

Metric

Score

Source

CodeSearchNet

MRR

67.2

Feng et al., 2020: Table 3

Clone Detection (BigCloneBench)

F1

96.5%

Feng et al., 2020: Table 5

## Performance

Input Size512 tokens max

Embedding Dim768

GPU Latency~3ms / snippet (A100)

CPU Latency~32ms / snippet

GPU Throughput~330 snippets/sec (A100)

GPU Memory~0.5 GB

## Common Pipeline Companions

[

Salesforce/codet5p-110m-embedding

Code embedding alternative

](/model/Salesforce/codet5p-110m-embedding)[

BAAI/bge-large-en-v1.5

Natural language query embedding

](/model/BAAI/bge-large-en-v1.5)

## Explore on Mixpeek

[

More Text Extraction models

Compare alternatives in this category

](/models?category=text_extraction)[

Best Multimodal RAG Frameworks

Hand-picked tools & platforms compared

](/curated-lists/best-multimodal-rag-frameworks)[

Multimodal RAG Pipeline Architecture

Deep-dive technical guide

](/guides/multimodal-rag-pipeline-architecture)[

Feature Extractors

See how Mixpeek runs models as extractors

](/extractors)[

Mixpeek Vector Store

Store & search embeddings at scale

](/mvs)[

Pricing

Usage-based pricing for pipelines

](/pricing)[

All Curated Lists

Compare models, APIs & infrastructure

](/curated-lists)

### Specification

FrameworkHF

Organizationmicrosoft

FeatureCode Extraction

Outputcode + language

Modalitiesdocument

RetrieverCode Search

Parameters125M

LicenseMIT

Downloads/mo221K

Likes287

[

### Research Paper

CodeBERT: A Pre-Trained Model for Programming and Natural Languages

arxiv.org](https://arxiv.org/abs/2002.08155)

### Build a pipeline with codebert-base

Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.

[Run it on your own data, free](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=model_page&utm_campaign=mvs_onramp)

### Alternative Models

[

Salesforce/codet5p-110m-embedding

Code Extraction

](/model/Salesforce/codet5p-110m-embedding)[

nomic-ai/nomic-embed-code

Code Extraction

](/model/nomic-ai/nomic-embed-code)[

Salesforce/SFR-Embedding-Code-2B\_R

Code Extraction

](/model/Salesforce/SFR-Embedding-Code-2B_R)[

jinaai/jina-code-embeddings-1.5b

Code Extraction

](/model/jinaai/jina-code-embeddings-1.5b)

### Related in Text Extraction

[

microsoft/trocr-large-printed

OCR

](/model/microsoft/trocr-large-printed)[

PaddlePaddle/paddleocr

OCR

](/model/PaddlePaddle/paddleocr)[

zai-org/GLM-OCR

OCR

](/model/zai-org/GLM-OCR)[

lightonai/LightOnOCR-2-1B

OCR

](/model/lightonai/LightOnOCR-2-1B)