fix(benchmark): repair 3 pre-existing script/download bugs (#534)

## Summary

Three unrelated pre-existing bugs surfaced while validating PR #515. All
of them block `Scripts/parakeet_subset_benchmark.sh --download` from
succeeding, but none are related to the v3 script-filtering work.
Consolidating into one PR since each fix is ~1–3 lines.

### 1. Japanese TDT folder-name mismatch

`Scripts/parakeet_subset_benchmark.sh` verifies the Japanese TDT model
at `$MODELS_DIR/parakeet-tdt-ja/`, but the folder was renamed to
`parakeet-ja` in 4ef33f0b6 (`Repo.parakeetJa.folderName =
"parakeet-ja"`). Result: `verify_assets()` always reported missing
assets even on a fully provisioned machine. One-line rename to match.

### 2. EOU streaming CLI writes to wrong path

`ParakeetEouCommand` had a default / `--use-cache` split where the
default branch produced `$CWD/Models/<chunk>/<chunk>/` (double-nested,
relative to CWD) as the load path, while `downloadModels()` called
`deletingLastPathComponent().deletingLastPathComponent()` then
`DownloadUtils.downloadRepo(repo, to:)` which appended `folderName =
"parakeet-eou-streaming/<chunk>"`. Net effect: files landed at
`$CWD/Models/parakeet-eou-streaming/<chunk>/` while `loadModels()`
looked at `$CWD/Models/<chunk>/<chunk>/` — model load failed silently.

Unified on Application Support (matches every other CoreML model in
FluidAudio). `--use-cache` retained as a no-op flag for backward
compatibility.

### 3. earnings22-kws dataset 404

HuggingFace consolidated `argmaxinc/earnings22-kws-golden` into
`argmaxinc/contextual-earnings22`. The old id now returns 404 from the
Datasets-Server REST API (no redirect follow). The new dataset has the
same feature schema (`audio`, `file_id`, `text`, `dictionary`, ...), so
swapping the id is sufficient — no downstream consumer changes needed.

## Test plan

Ran `Scripts/parakeet_subset_benchmark.sh --download` end-to-end:

- [x] `verify_assets` correctly resolves `parakeet-ja/` (all 5 expected
files present)
- [x] EOU warmup: `Models downloaded to ~/Library/Application
Support/FluidAudio/Models/parakeet-eou-streaming/320ms`, 0.00% WER on
warmup file
- [x] earnings22-kws: 1140+ files downloaded (was 0 before), no 404
- [x] `swift build` passes

Out of scope but observed (pre-existing, unrelated):
- `ctc-earnings-benchmark --auto-download` does not actually
auto-download CTC-110m model
- THCHS-30 dataset hit HF IP rate limit (429) — transient
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/fluidinference/fluidaudio/pull/534"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open in Devin Review">
  </picture>
</a>
<!-- devin-review-badge-end -->
This commit is contained in:
Alex
2026-04-21 04:22:18 -04:00
committed by GitHub
parent f8badf7899
commit 7c9be31c05
3 changed files with 10 additions and 13 deletions
+2 -2
View File
@@ -8,7 +8,7 @@
# 4. CTC custom vocab — ctc-earnings-benchmark (v2 TDT + CTC 110m keyword spotting)
# 5. EOU streaming — parakeet-eou 320ms on LibriSpeech test-clean
# 6. Nemotron streaming — nemotron 1120ms on LibriSpeech test-clean
# 7. TDT Japanese — parakeet-tdt-ja on JSUT dataset
# 7. TDT Japanese — parakeet-ja on JSUT dataset
# 8. CTC Chinese — parakeet-ctc-zh-cn on THCHS-30 dataset
#
# Usage:
@@ -103,7 +103,7 @@ verify_assets() {
# Nemotron reuses the v3 models directory; no separate check needed beyond v3 above.
# --- Japanese TDT (hybrid: CTC preprocessor/encoder + TDT decoder/joint) ---
local ja_dir="$MODELS_DIR/parakeet-tdt-ja"
local ja_dir="$MODELS_DIR/parakeet-ja"
for f in Preprocessor.mlmodelc Encoder.mlmodelc Decoderv2.mlmodelc Jointerv2.mlmodelc vocab.json; do
if [[ ! -e "$ja_dir/$f" ]]; then
log "MISSING tdt-ja: $ja_dir/$f"