LateOn-Code
by lightonai
ColBERT-style late interaction model purpose-built for code retrieval
lightonai/LateOn-Codemixpeek://text_extractor@v1/lighton_lateon_code_v1Overview
LateOn-Code is a late-interaction (ColBERT-style) embedding model specifically designed for code retrieval. Unlike dense single-vector code embedders that compress an entire function into one vector, LateOn-Code preserves token-level information through multi-vector representations — enabling precise matching of variable names, API calls, and code patterns.
On Mixpeek, LateOn-Code enables semantic code search across codebases — find functions by describing what they do, locate similar implementations across repositories, or search for specific API usage patterns. The 149M parameter size makes it deployable alongside heavier models without significant overhead.
Architecture
ModernBERT backbone fine-tuned for ColBERT-style multi-vector output. 149M parameters. Produces per-token embeddings for late interaction scoring. Also available as a 17M edge variant (LateOn-Code-edge) that outperforms models 3x its size.
Mixpeek SDK Integration
import { Mixpeek } from "mixpeek";const mx = new Mixpeek({ apiKey: "API_KEY" });await mx.collections.ingest({collection_id: "codebase",source: { url: "https://example.com/repo.tar.gz" },feature_extractors: [{feature: "code_embedding",model: "lightonai/LateOn-Code"}]});
Capabilities
- SOTA code retrieval on MTEB Code v1 (74.12 nDCG avg)
- ColBERT-style token-level matching for precise code search
- Understands variable names, API calls, and code structure
- 149M params — lightweight enough for real-time search
- Apache 2.0 license
Use Cases on Mixpeek
Benchmarks
| Dataset | Metric | Score | Source |
|---|---|---|---|
| MTEB Code v1 | nDCG avg | 74.12 | LightOn, 2026 |
| BEIR | nDCG@10 | >57.0 | LightOn, 2026 |
Performance
Specification
Research Paper
LateOn-Code: ColBERT for Code Retrieval
arxiv.orgBuild a pipeline with LateOn-Code
Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.
Open Studio