uzasr 0.1.0: open Uzbek speech recognition built on GigaAM

In short. We released two Uzbek speech recognition models, uzasr-large (600M parameters) and uzasr-base (220M). Both are fine-tuned from GigaAM-Multilingual and output punctuated text. On our spontaneous-speech benchmark, uzasr-large scores 12.2% WER. The untuned base model scores 16.1%, and the best public Whisper fine-tune for Uzbek scores 39.6%. The weights, configs and benchmark are public on Hugging Face.

Why we didn't start from Whisper

We began with the obvious candidates: Whisper large-v3 and its public Uzbek fine-tunes. On real Uzbek speech (podcasts, interviews, lectures) they had three recurring failures:

• Words drifted into neighbouring Turkic languages.
• Whole phrases were dropped.
• Text was hallucinated over silence.

A side-by-side listening test across channels favoured GigaAM-Multilingual, an open (MIT) CTC model whose pretraining covers Uzbek. A CTC model is also far less prone to inventing text over silence, which matters for production transcripts.

What we built

• Two sizes, one recipe. We found the recipe on the 220M model first, where experiments are cheap, then trained the 600M model with the same recipe. On the 220M model, data returns ran out: the first 120 hours of new spontaneous speech gave −1.0 point WER, and the next 540 hours gave almost nothing. Model capacity was the remaining lever. The 600M model reached a better validation score in 4,000 steps than the 220M model did in 22,000.
• Punctuation inside the model. We added . , ? ! to the CTC vocabulary and trained on corpora that kept their original punctuation, so the model infers punctuation from pauses and intonation. We did not use a separate punctuation model. Sentence-initial capitalisation is applied by a deterministic rule.
• Training data. Open Uzbek corpora (Common Voice, FLEURS, UzbekVoice, USC, curated YouTube sets) plus spontaneous YouTube speech with machine transcripts. Every source was license-checked: Apache-2.0, MIT, CC0 or CC-BY.

Problem: no trustworthy test set

Public Uzbek test sets are mostly read speech. On FLEURS, the untuned model already reached 6.7% WER, which says little about how it handles a podcast. We needed a test set of the speech our users actually send.

Our first attempt was hand-correcting our own transcripts. It stalled after about 40 segments because the error density made each one slow.

Solution. We built the benchmark from a corpus with strong machine transcripts and checked it against a second model. In total it has 311 clips, 1.98 hours of spontaneous speech.

• For each clip we measured disagreement between the transcript and GigaAM-large. A human reviewed all 121 clips with disagreement above 18%, starting from the largest.
• The 194 low-disagreement clips were accepted as-is. A random spot-check of that group found no errors.
• The benchmark is public. To respect the original data licence it contains text only; a script rebuilds the audio from the upstream dataset.

Problem: spelling conventions counted as errors

About 10% of substitutions were not recognition errors. They were the same word written two ways, such as keldimi / keldi mi or beshta / besh ta.

Solution. Before scoring, we apply one deterministic rule to both sides that joins standalone clitics (mi, da, ku, chi, ta, ya, yu). We report both the plain and the canonical WER. Expected impact was 3–4 points; the measured impact was 0.6. We publish the measured number.

Results (benchmark v2)

What we got wrong

We trained the 600M run without writing a run log as we went. When the local run directory was lost, we had to rebuild the training history from the metadata stored inside the checkpoints. The run was also stopped at step 6,000 of 8,000, so this number is a floor, not a ceiling. Since then, every run gets an evaluation note the day it finishes.

Next

Our internal target is under 10% WER on this benchmark. The plan is a language model in the decoder, training on more spontaneous speech, and then speaker diarization.