WAVE-7B

Name: WAVE-7B
Author: tsinghua-ee

by tsinghua-ee

Unified audio-visual embeddings for text, audio, silent video, and synchronized clips

288dl/month

6likes

7Bparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

tsinghua-ee/WAVE-7B

Feature URI

mixpeek://audio_extractor@v1/tsinghua_wave_7b_v1

Deploy WAVE-7B

Single-tenant

Mixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.

Read the customer-hosted guide Talk to us about single-tenant

Overview

WAVE 7B is a Qwen2.5-Omni based embedding model for unified audio-visual retrieval. It creates a shared representation space for text, audio, silent video, and synchronized audio-video inputs, with prompt-aware embeddings for instruction-specific retrieval.

On Mixpeek, WAVE is a strong candidate when agents need to search multimodal observations where sound and motion both matter. A support agent can retrieve the clip where a machine squeals before stopping; a media agent can find a scene by its crowd sound and camera motion; an inspection agent can search for audiovisual anomalies without relying on transcripts alone.

Architecture

7B-class multimodal embedding model built on Qwen2.5-Omni with hierarchical feature fusion and a dual audio encoder for speech and environmental sound. It is trained with multimodal, multitask contrastive objectives across text, audio, video, and audio-video pairs.

Mixpeek SDK Integration

// No extractor parameter takes a Hugging Face model id (checked against
// GET /v1/discovery/extractors, which returns 13), so WAVE-7B runs
// on your side and the output is upserted through POST
// /v1/namespaces/{namespace_id}/documents/upsert. On Enterprise the other
// path is to upload the weights instead: POST /v1/namespaces/{id}/models
// accepts the huggingface format and a custom plugin loads them.
const res = await fetch(
  "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer API_KEY",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      collection_id: "col_your_collection",
      documents: [
        {
          document_id: "asset-00412",
          // The vector name has to match a vector index on the collection.
          vectors: { "audio-embedding": yourVector },
          payload: { source_key: "archive/2026/asset-00412" },
        },
      ],
    }),
  },
);

// Managed alternative, if this exact model is not the requirement:
// audio_fingerprint_extractor@v1 runs laion/clap-htsat-tiny
// (512-d) over a bucket, with no inference of your own.