mirror of https://github.com/FluidInference/FluidAudio.git synced 2026-05-12 20:20:36 +00:00

Files

T

Brandon Weng 7fd5ac5446 pyannote community-1 model for offline speaker diarization pipeline (#150 )

### Why is this change needed?
<!-- Explain the motivation for this change. What problem does it solve?
-->

Keeping the streaming one around as the VBx and AHC clustering gets
pretty expensive after 30mins of audio and running it constantly gets
expensive. Its still possible to support clustering between files but
will save that for another PR.

Pyannote's Bench mark is around 11% - i increased steps to 0.2s instead
of 0.1 to double the speed but also selective fp16 results in more
operations to run on ANE but also means that we lose some precision.

```
Average DER: 14.95% | Median DER: 10.89% | Average JER: 39.27% | Median JER: 40.74% (collar=0.25s, ignoreOverlap=True)
Average RTFx: 139.63 (from 232 clips)
Metrics summary saved to: /Users/brandonweng/FluidAudioDatasets/voxconverse/metrics/test_metrics_release.json
Completed. New results: 232, Skipped existing: 0, Total attempted: 232
```

See benchmark.md for more info but compared to Pytorch model, we are
100x faster than the CPU version and ~6x faster compared to the mps
backend on mb pro 4

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Brandon Weng <BrandonWeng@users.noreply.github.com>
Co-authored-by: Alex <36247722+Alex-Wengg@users.noreply.github.com>
Co-authored-by: Alex-Wengg <hanweng9@gmail.com>

2025-10-22 15:11:57 -04:00

2.1 KiB

Raw Blame History

FastCluster Wrapper

This directory contains a C wrapper around the fastcluster library, specifically exposing centroid linkage hierarchical clustering for Swift.

Purpose

The FastCluster wrapper is required for accurate reimplementation of the pyannote community-1 speaker diarization pipeline in Swift. The pyannote pipeline uses agglomerative hierarchical clustering with centroid linkage to cluster speaker embeddings, and this wrapper provides an efficient C++ implementation via a C interface accessible from Swift.

What's Included

FastClusterWrapper.cpp: C wrapper implementation
fastcluster_internal.hpp: Internal fastcluster algorithms (from upstream fastcluster)
include/FastClusterWrapper.h: C API header
include/module.modulemap: Swift module bridge

Functionality

`fastcluster_compute_centroid_linkage()`

fastcluster_wrapper_status fastcluster_compute_centroid_linkage(
    const double *data,          // Feature vectors (row-major layout)
    size_t pointCount,           // Number of vectors
    size_t dimension,            // Feature dimension
    double *dendrogramOut,       // Output dendrogram (SciPy format)
    size_t dendrogramLength      // Output buffer size
);

Computes agglomerative hierarchical clustering using centroid linkage on the input feature vectors. Returns a dendrogram in SciPy format (4 columns: left node, right node, distance, sample count).

Integration

Used by Sources/FluidAudio/Diarizer/Offline/AHCClustering.swift to perform speaker embedding clustering, which is a core component of the diarization pipeline.

Source

Original Repository: https://github.com/fastcluster/fastcluster
Algorithm: Centroid linkage hierarchical clustering
Reference: Based on algorithms by Daniel Müllner and Google Inc.

License

fastcluster is licensed under the BSD 2-Clause License. See ThirdPartyLicenses/fastcluster-LICENSE.md for details.

2.1 KiB Raw Blame History