NEWVectors or files. Pick a path.Start →
    Models/Speech & Audio/pyannote/wespeaker-voxceleb-resnet34-LM
    HFSpeaker DiarizationCC-BY-4.0

    wespeaker-voxceleb-resnet34-LM

    by pyannote

    WeSpeaker's ResNet34 speaker embeddings, wrapped for pyannote.audio

    5.7Mdl/month
    Not stated on the model cardparams
    Identifiers
    Model ID
    pyannote/wespeaker-voxceleb-resnet34-LM
    Feature URI
    mixpeek://transcription@v1/pyannote_wespeaker_resnet34_lm_v1

    Deploy WeSpeaker ResNet34-LM

    Single-tenant

    Mixpeek has no managed extractor that runs these weights. On a single-tenant deployment you upload them and a custom plugin serves them next to the rest of your pipeline.

    Overview

    These are the WeSpeaker toolkit's ResNet34 weights packaged so pyannote.audio 3.1 can load them directly. The LM suffix is the large-margin fine-tuning stage applied after the main training run, and it is what separates this checkpoint from the plain ResNet34 release.

    Training is on the VoxCeleb2 development set, 5,994 speakers. The WeSpeaker model zoo reports 0.723% EER on vox1-O-clean with large-margin fine-tuning and AS-Norm applied, 0.867% on vox1-E-clean and 1.532% on vox1-H-clean. The wrapper is CC-BY-4.0, so attribution is a condition of use.

    It does the same job as ECAPA-TDNN and takes the same place in a pipeline: audio in, one vector per segment out, cosine distance to compare them. Which one wins on your corpus is a question for your corpus.

    Architecture

    ResNet34 with [3, 4, 6, 3] residual blocks over an 80-bin mel spectrogram, temporal statistics pooling, and a 256-dimensional embedding (the zoo names the configuration ResNet34-TSTP-emb256). Training runs the usual margin-based classification objective and then a large-margin fine-tuning pass; the published figures also apply AS-Norm score normalisation at test time, which is a scoring step rather than part of the model.

    Mixpeek SDK Integration

    // No extractor runs these weights, so the model runs on your side and the
    // embedding is upserted through POST /v1/namespaces/{namespace_id}/documents/upsert.
    // On a single-tenant deployment the other path is to upload the weights:
    // POST /v1/namespaces/{id}/models accepts the huggingface format and a custom
    // plugin loads them.
    const res = await fetch(
      "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
      {
        method: "POST",
        headers: {
          Authorization: "Bearer API_KEY",
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          collection_id: "col_call_segments",
          documents: [
            {
              document_id: "call-00412-seg-07",
              // 256 dimensions here, against 192 for ECAPA-TDNN. The vector index
              // has to match, so running both means two indexes, not one.
              payload: {
                transcript: segment.text,
                speaker_id: match.speakerId,
                speaker_score: match.cosine,
              },
              vectors: { "speaker-embedding-256": wespeakerEmbedding },
            },
          ],
        }),
      },
    );

    Capabilities

    • 256-dimensional speaker embedding per segment
    • Loads straight into pyannote.audio 3.1 as a drop-in embedding model
    • Speaker verification and open-set identification by cosine distance
    • Large-margin fine-tuned checkpoint, which is the stronger of the two WeSpeaker ResNet34 releases

    Use Cases on Mixpeek

    Swap the embedding model inside an existing pyannote diarization pipeline
    Score a second recognizer against your first before committing to one
    Per-speaker retrieval over meetings, calls and interviews
    Voice-based deduplication across an archive with unreliable metadata

    Benchmarks

    DatasetMetricScoreSource
    VoxCeleb1-O (clean)EER0.723%WeSpeaker VoxCeleb model zoo, ResNet34-TSTP-emb256 with large-margin fine-tuning and AS-Norm
    VoxCeleb1-E (clean)EER0.867%WeSpeaker VoxCeleb model zoo, same configuration
    VoxCeleb1-H (clean)EER1.532%WeSpeaker VoxCeleb model zoo, same configuration

    Performance

    Input Sizemel spectrogram, 80 bins, variable length
    Embedding Dim256
    GPU LatencyInput dependent
    GPU ThroughputBatch dependent
    GPU MemoryModel dependent

    We have not published our own measurement on Mixpeek infrastructure, so no latency figure is quoted here. The published EER figures include AS-Norm, which needs a cohort at scoring time; without it expect them to be a little worse.

    Frequently Asked Questions

    What does the LM in the name mean?

    Large margin. It is a fine-tuning pass run after the main training, on longer crops with a bigger angular margin, and it is why this checkpoint scores better than the plain ResNet34 release in the same zoo.

    Can I use it commercially?

    Yes, under CC-BY-4.0, which requires attribution. That is a lighter condition than a share-alike or non-commercial licence, and it is stricter than the Apache-2.0 on ECAPA-TDNN, so check it against your own policy before shipping.

    Is it interchangeable with ECAPA-TDNN?

    Functionally yes, dimensionally no. This returns 256 numbers and ECAPA-TDNN returns 192, so the vector index is configured for one or the other and switching means re-embedding rather than an in-place write.

    Specification

    FrameworkHF
    Organizationpyannote
    FeatureSpeaker Diarization
    Outputspeaker segments
    Modalitiesvideo, audio
    RetrieverSpeaker Filter
    ParametersNot stated on the model card
    LicenseCC-BY-4.0
    Downloads/mo5.7M

    Research Paper

    Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit

    arxiv.org

    Build a pipeline with wespeaker-voxceleb-resnet34-LM

    Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.

    Run it on your own data, free