Uzbek speech recognition benchmark
We measure word error rate (WER) on 311 clips of spontaneous Uzbek speech from podcasts, interviews and multi-speaker conversations, with reference transcripts checked by hand. uzasr-large gets 8.1% of words wrong, untuned GigaAM-large 14.4%, and whisper-large-v3-turbo-uzbek 38.6%. The test set, the scoring script and every model's predictions are open on Hugging Face.
Updated Oct 28, 2026
Results
Every model was run on the same audio and scored with the same script.
| Model | Parameters | WER | CER |
|---|---|---|---|
| uzasr-largeClarion | 600M | 8.1% | 2.5% |
| uzasr-baseClarion | 220M | 9.4% | 3.0% |
| GigaAM-large, not fine-tuned | 600M | 14.4% | 5.3% |
| whisper-large-v3-turbo-uzbek | 809M | 38.6% | 18.5% |
WER is the share of misrecognized words. Lower is better. Punctuation is not counted.
What's in the test set
Real conversation, not read speech.
- Clips
- 311
- Hours of audio
- ≈2
- Disputed spots checked by ear
- 374
- Clips corrected after listening
- 144
Spontaneous Uzbek speech from YouTube: podcasts, interviews and conversations between several speakers. People talk over each other, hesitate, switch into Russian and use dialect forms.
The source transcripts were made by a machine. We compared each one with the output of a second model, GigaAM-large, and a person reviewed word by word every clip where the two disagreed strongly. A random spot check of the remaining clips found no errors.
Later a native speaker listened to the audio wherever our newest model and the reference disagreed, and decided which one was right. An LLM only sorted the disputed spots by priority. Every change to the reference comes from a human decision.
Method
How WER and CER are computed, and where the method is biased.
WER is the number of substituted, deleted and inserted words divided by the number of words in the reference. CER is the same count over characters. Both are computed over the whole set at once, not averaged per clip.
Before scoring, both texts are lowercased, apostrophes are brought to one form and punctuation is removed, so punctuation does not affect the result. Quotation marks are dropped; an apostrophe is kept only between two letters.
Clitics such as mi and da are written both joined and apart in real Uzbek text. The dataset card also reports a variant that joins them on both sides; this page shows the plain WER.
Every release of our models and every third-party model is scored on the same clips with the same script. Per-clip predictions for each model in the table are published.
Bias in the method
We re-checked the reference only where our newest model, uzasr-large v0.3.0, disagreed with it. So the corrections fix exactly the errors that hurt that model most: it gained 2.6 WER points, our other models 1.4 to 2.1, GigaAM-large 1.7 and Whisper 0.4. The ranking did not change, but the gap between our own models looks somewhat larger than it is. Each new model's disagreements go through the same listening review before we publish its number.
Accuracy by release
From the untuned base model to the current uzasr-large, every step scored on the same clips and the same reference.
| Version | Change | WER |
|---|---|---|
| GigaAM-large | untuned base | 14.4% |
| v0.1.0 | fine-tuned | 10.4% |
| v0.2.0 | withdrawn | — |
| v0.2.1 | + language model | 9.4% |
| v0.3.0 | domain training | 8.1% |
Corrections we published
When a published number turned out to be wrong, we said so.
0.2.0 was withdrawn. Its language model had been trained on text that overlapped 62 of the 311 test clips, so the published WER was about 1.3 points too low.
Read the post →0.2.1 rebuilt the language model with the overlapping text removed and published the honest WER. The acoustic model was not affected.
Read the post →
Reproduce it
Everything needed to check our numbers or score your own model.
The dataset contains the reference transcripts (current and previous version), every correction and listening decision, per-clip predictions for each model in the table, and the scoring script.
Audio is not included: a script rebuilds it from the open upstream dataset the clips come from.
The annotations are released for research and evaluation use. The audio belongs to its original YouTube publishers and is not redistributed; the upstream dataset's terms apply to it.