Kyutai STT 2.6B English

Kyutai/stt-2.6b-en is a streaming automatic speech recognition (ASR) model that transcribes English audio into text with punctuation and capitalization, producing output as soon as a few seconds of audio become available.

Key Capabilities

• Streaming inference: processes audio in chunks for real-time transcription, suitable for interactive applications.
• Returns word-level timestamps for each transcribed token.
• Robust to noisy conditions; performs reliably on audio segments up to 2 hours without additional adaptation.
• Based on a decoder-only Transformer architecture that consumes audio tokenized by Mimi (12.5 Hz frame rate, 32 audio tokens per frame) and outputs text tokens. The text stream is shifted by a 2.5-second delay relative to the audio stream.

Performance

• On a single H100 GPU, the model can batch-process 400 audio streams in real time.
• A single L40S GPU serves 64 simultaneous streaming connections via a Rust websocket server at a 3x real-time factor.

Training Details

The model was pretrained on 2.5 million hours of publicly available audio with synthetic transcripts from Whisper-timestamped, then fine-tuned on 24,000 hours of ground-truth transcribed public datasets, followed by a long-form fine-tuning stage using concatenated LibriSpeech examples and synthesized dialogs (total 23,000 hours).

Additional Features

The model outputs transcripts that include capitalization and punctuation. Word-level timestamps can be derived by subtracting the 2.5-second text stream offset from the audio frame offset. The model is English-only (language identifier en) and released under CC-BY 4.0. Parameter count is approximately 2.7 billion (verified via safetensors metadata).