CLAP2-L
by laion
LAION's open audio-text embedding model: 340M parameters, one 960-dimension vector per clip, trained from scratch on 366M audio-text pairs
laion/CLAP2-LDeploy CLAP2-L
Single-tenantMixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.
Overview
CLAP2-L is the largest of LAION's five CLAP2 contrastive language-audio models, published on Hugging Face on 26 October 2026 under the Apache 2.0 license. An audio tower and an English text tower map sound clips and written descriptions into one 960-dimension space, so you can search an audio library by describing what you want to hear, classify sounds with no training labels, or use the audio vectors as general features. All five sizes, from 27M to 340M parameters, were trained from scratch on LAION-Audio-XL, 366 million audio-text pairs of at most 30 seconds.
Architecture
A two-tower contrastive model in OpenCLIP and Transformers format, 340.1M parameters across both towers. The audio tower is a NaFlex vision transformer over 128-bin log-mel patches (16x16, 50 tokens per second at 32 kHz), so it takes clips of any length with no fixed crop. The text tower reads English through a case-sensitive GPT-2 style tokenizer with a 256-token context. Training saw 1,024M samples with a warmup-stable-decay schedule; the release is the last checkpoint.
Mixpeek SDK Integration
# Embed audio clips (usage from the model card).
# pip install "open_clip_torch>=4.2.1" soundfile
import torch, open_clip
from open_clip.audio.io import load_audio
hub = "hf-hub:laion/clap2-large-audioxl-s1B"
model, preprocess = open_clip.create_model_from_pretrained(hub)
tokenizer = open_clip.get_tokenizer(hub)
audio = preprocess()(load_audio("scene_0042.wav")) # any sample rate, any channels
with torch.no_grad():
clip_vector = model.encode_audio({k: v[None] for k, v in audio.items()}, normalize=True)
# Store the 960-d vector with the file and timestamp it came from.
# Split recordings longer than 30 s into segments first.Capabilities
- Text-to-audio search: describe a sound and find matching clips
- Zero-shot audio classification with written class descriptions
- Audio embeddings for clustering, deduplication and downstream classifiers
- Clips of any sample rate, channel count and length, embedded without padding
- Five sizes, from 27.3M to 340.1M parameters, with the same interface
Use Cases on Mixpeek
Benchmarks
| Dataset | Metric | Score | Source |
|---|---|---|---|
| MAEB-30 | Mean over 30 tasks | 53.6 | LAION model card; CLAP2-M 52.3 and CLAP2-XXS 46.2 on the same table |
| HEAR-17 | Mean over 17 tasks | 81.9 | LAION model card |
| MSEB | Mean over 62 English scores | 30.3 | LAION model card |
Performance
We have not measured it. The scores are LAION's own, from the model card and the paper (arXiv:2611.11592). Cost grows with the square of clip length, so split long recordings into segments of up to 30 seconds, as the card recommends.
Common Pipeline Companions
Frequently Asked Questions
What is CLAP2-L?
An open contrastive language-audio model from LAION, published on 26 October 2026 under Apache 2.0. It has 340M parameters and embeds audio clips and English text into one 960-dimension space for text-to-audio search, zero-shot audio classification and general audio embeddings. The card reports 53.6 on MAEB-30 and 81.9 on HEAR-17.
How do I search audio by describing a sound?
Embed every clip with the audio tower, embed your description with the text tower, and rank clips by cosine similarity. Write the description as a full sentence in normal case, such as "A dog barks.", which matches the style of the training captions better than a bare label.
Can CLAP2 handle long recordings?
It accepts any length, but it was trained and evaluated on clips of at most 30 seconds and its cost grows with the square of the length. The card recommends splitting long recordings into segments of up to 30 seconds and averaging or comparing the segment embeddings.
Which CLAP2 size should I use?
Start with the smallest that meets your accuracy bar. The five sizes run from 27.3M to 340.1M parameters, and the card's MAEB-30 means range from 46.2 to 53.6, with the 103.9M small model at 51.8, close to the large one at a third of the size.
Does CLAP2 understand speech?
It embeds how a clip sounds and does not transcribe it. About 45% of its training audio is speech, and the card's own example captions include "A man gives a lecture.", so a description of the speaker and setting can match. Use speech-to-text for what was said and CLAP2 for what was heard, and search both.
Specification
Build a pipeline with CLAP2-L
Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.
Run it on your own data, free