Dutch Open Speech Recognition Benchmark

Welcome to the Dutch Open Speech Recognition Benchmark website! We encourage contributions from all researchers and developers working with Dutch Automatic Speech Recognition.

OH-SMArt Project Benchmark

The following results were achieved during the PDI-SSH Oral History - Stories at the Museum around Art (OH-SMArt) project (2022-2025) and are reported by University of Twente:

Broadcast News and Telephone Conversations (N-Best) Benchmark

Underrepresented Speakers (JASMIN-CGN) Benchmark

Common Voice (CV) Benchmark

Pathological Speech (COPAS) Benchmark*

*These results were obtained during and after the project, in preparation for Interspeech 2025.

Environment setup for benchmarks above

Why do the results differ between whisper-timestamped and faster-whisper?

Medical Speech (HoMed) Benchmark

The following results were achieved during the PDI-SSH Homo Medicinalis (HoMed) project (2021-2024) and are reported by Radboud University:

Results

Environment setup

NISV’s Whisper Benchmark

NISV = Netherlands Institute for Sound & Vision (NL: Nederlands Instituut voor Beeld & Geluid)

The following results were achieved during the same (OH-SMArt) project mentioned above and are reported by University of Twente in collaboration with NISV:

Results for Broadcast News Speech

Results for Conversational Telephone Speech

Vlotspraak’s Benchmark

Vlotspraak = a proprietary, on-premise commercial Dutch ASR model, developed by CodeSpark Tech

The following results were produced by CodeSpark Tech and scored with the official ASR_NL_benchmark Docker tool (text normalization + NIST sclite). The model weights are commercial and not publicly released, but the full sclite output is committed so the scoring itself is inspectable.

Common Voice (nl) and Medicijnjournaal (HoMed) Benchmarks

Common Voice 17.0 NL test: 3.5% WER on the full 11,266-clip set. A same-model faster-whisper-large-v3 control on the identical clips reproduces the 4.3% figure reported for that model on this board, and a train-on-test contamination check is documented (0.08% overlap).

Held-out medical audio, no medical data in training: 11.4% WER on 30 of the 35 HoMed Medicijnjournaal episodes (the audio we could retrieve). This is a different subset than the 35-file RU evaluation, so it is indicative rather than a like-for-like comparison - see the page for the full caveat.

Environment setup

Contributions

Feel free to click the link at the top that leads you to the GitHub repository of this website. You may add changes if you want by forking the repository, making changes on your fork, then opening a pull request on the source repository.