Served on MixpeekEmbeddingVerified 2026-11-05
LAION CLAP HTSAT-tiny (630k-audioset-best)
lukewys/laion_clap
LAION CLAP HTSAT-tiny (630k-audioset-best) is a audio and text embedding model (CC0-1.0). Served on Mixpeek, scaling to zero when idle.
A Ray Serve app in Mixpeek's inference registry hosts it. It starts at zero replicas and wakes on first use.
New opens the Designer in Studio. For an existing namespace, open it in Studio and use its Designer tab.
No demo for LAION CLAP HTSAT-tiny (630k-audioset-best) yet
A video, image, slideshow or playground goes here once we record one on our own data.
What it is good and bad at
- Best at
- Sound-alike search over short audio windows (jingles, music beds, effects) and text-to-audio search ("upbeat jingle with bells") in one 512-d space.
- Worst at
- Speech content (use whisper + E5); exact duplicate detection where an acoustic fingerprint algorithm is cheaper; audio longer than the 5 s window is seen only as overlapping windows.
- Task
- Embedding
- Input
- Audio, Text
- Output
- Vector
- Use cases
- Audio search
Templates that use it
No template uses this model yet. Browse templates.
Input and output
Input
- audio
- sample_rate
- 48000
- window_s
- 5
- hop_s
- 2.5
- channels
- mono
- text
- free-text description of a sound
Output
- vector
- dims
- 512
- distance
- cosine
- type
- dense
How Mixpeek wires it
- Extractors
audio_fingerprint_extractor@v1fixed
- Feature URIs
mixpeek://audio_fingerprint_extractor@v1/laion__clap_htsat_tiny
- Vector indexes
audio_fingerprint_extractor_v1_embedding- Retriever stages
- feature_search
- Combines with
- audio or video -> 5 s windows with 2.5 s hop -> CLAP vector per window -> feature_search by audio clip or text
How it runs
- Status
- Served on Mixpeek Ray Serve app in the inference registry (query embeds); audio_fingerprint_extractor loads the same checkpoint in-process (AudioFingerprintBatchService).
- Serve apps
laion__clap_htsat_tiny- Runtime
- laion_clap.CLAP_Module(enable_fusion=False) + load_ckpt (HTSAT-tiny audio encoder, RoBERTa text encoder)
- Weights
- Pickle, no remote code
- Parameters
- Not recorded
- Size on disk
- 1,864 MB
- Preprocessing
- Extract audio from video, resample to 48 kHz, slice 5 s windows every 2.5 s; text goes through the CLAP text tower.
- Postprocessing
- get_audio_embedding_from_filelist / get_text_embedding (512 dims), normalized when normalize_embeddings is true.
- Serving
- both, batch 8
- Hardware
- Runs on CPU (slower without a GPU)
Scale to zero
- Minimum replicas
- 0
- Maximum replicas
- 4
- Idle timeout
- 600 s
- Memory
- 2 GB
- CPUs
- 1
- GPUs
- 0
- Cold start, node warm
- Not recorded
- Cold start, new node
- Not recorded
Measured cost
Unmeasured
No committed throughput or cost measurement for this model.
Our evaluations
Unmeasured
We have not evaluated this model on our own data yet. Publisher results are in the sources below.
Questions
- What is LAION CLAP HTSAT-tiny (630k-audioset-best) best at?
- Sound-alike search over short audio windows (jingles, music beds, effects) and text-to-audio search ("upbeat jingle with bells") in one 512-d space.
- What are the limits of LAION CLAP HTSAT-tiny (630k-audioset-best)?
- Speech content (use whisper + E5); exact duplicate detection where an acoustic fingerprint algorithm is cheaper; audio longer than the 5 s window is seen only as overlapping windows.
- Can LAION CLAP HTSAT-tiny (630k-audioset-best) be used commercially?
- Yes. The checkpoint repo is CC0-1.0 (public domain dedication). Training data includes AudioSet and LAION-Audio-630K; no license restriction on the weights.
- How does LAION CLAP HTSAT-tiny (630k-audioset-best) run on Mixpeek?
- Ray Serve app in the inference registry (query embeds); audio_fingerprint_extractor loads the same checkpoint in-process (AudioFingerprintBatchService).