Nvidia has released Nemotron 3.5 Transcribe, a free open-weight model capable of transcribing audio in up to 40 languages.
Version 2 of Granary, Nvidia's open source speech dataset, is now available. The company presents it as a transcription specialist, not a general-purpose assistant. Its technical research report says a transcript has no single correct form: numbers, fillers and domain terms depend on who'll read it.
Technical architecture and parameters
What we know
- Nemotron 3.5 Transcribe is a speech-augmented language model (SALM) in which an audio encoder feeds embeddings into an LLM decoder.
- The model has 30 billion parameters, but only about 4 billion are active per processing step.
- A text instruction picks the output style for each request.
Still unclear
- Whether the serving optimizations that produce the fastest transcription speeds will be open sourced, and when.
Transcription style can be set with a prompt. For each request, users give a text direction specifying whether they want verbatim, written form, speaker-attributed, contextual or translated output.
The model is named 30B-A4B and has about 34 billion parameters, including a 1.3 billion MTP module and the audio components.
Performance and benchmarks
Nvidia said the model runs on one chip, including its Jetson AGX Thor robotics controller, DGX Spark chip and Blackwell datacenter chips. With NVFP4 precision, an RTX PRO 6000 Blackwell Server Edition can process 100 hours of audio in less than three minutes.
With NVFP4 and DFlash2, one RTX PRO 6000 Blackwell Server Edition handles 2,238 seconds of audio in a second. It takes 0.13 seconds to process a 30-second clip. Nvidia expects the release-day vLLM path to be roughly twice as slow as its optimized serving path, and estimates about 410 RTFx on one H100 in BF16 with MTP2.
Nemotron 3.5 Transcribe
Open-weight model
- 5.33% average word error rate on Hugging Face English Open ASR Leaderboard
- 2.81% score in Artificial Analysis comparison
- First place in VoiceArena overlapping speech tests
- Lowest error rate on David AI speaker attribution benchmark
Competitors
Voxtral Small and others
- Voxtral Small scores 2.77% in Artificial Analysis
- VoiceArena median misses about 31% of words
- Open medical speech model scores 13.79% on David AI benchmark
Transcribe ranked first among open-weight systems on Hugging Face's English Open ASR Leaderboard, posting a 5.33 percent average word error rate. Artificial Analysis measured its score at 2.81 percent, near Voxtral Small, the top open model at 2.77 percent, despite using only about a sixth of the active parameters.
On conversations with occasional overlapping speech, VoiceArena evaluated 14 systems, with Transcribe coming first. Its word error rate was 8.32 percent, the lowest of the systems, against a median of about 31 percent. David AI's benchmark measures speaker attribution across 21 languages. There, the model recorded the lowest error rate, narrowly beating Nvidia's own dedicated model for the task.
Training data and fine-tuning
Training involved one supervised fine-tuning stage, using 256 H100 GPUs for 28,500 steps, followed by 400 GSPO updates in a short RL stage. The reward was 1 minus WER. The model trained on about 1.72 million unique hours of audio, equivalent to 6.5 million hours with repeats.
Granary v2 contains about 463,000 hours of speech across 48 languages, compared with 25 languages in the previous version. Recordings have multiple transcription styles: verbatim transcripts retain fillers such as "uh" and spell out numbers, while cleaned-up versions omit fillers and use digits.
Developers can process their own recordings with the tools Nvidia is making available to create the dataset. The company said the dataset doesn't fully match the model's training data.
Using recordings made by doctors and medical students in India, Nvidia tested targeted fine-tuning. Across the full benchmark, refining the prompt reduced the error rate from 30.8 to 23.8 percent, while fine-tuning lowered it further to 11.8 percent. For previously unheard speakers, Nvidia reported an error rate of 12.17 percent, compared with 22.44 percent, a 45.8 percent relative improvement. The adapted model outscored an open medical speech model, which scored 13.79 percent.
Medical fine-tuning took about 14 GPU hours on one Nvidia GB300. Specialization slightly increased the error rate on general English speech, Nvidia said. For custom versions, it recommends using at least 100 real, verified recordings as a test set.
Availability and limitations
Weights are available in BF16, FP8, NVFP4 and GGUF, and users can add a DFlash2 draft model. The OpenMDW-1.1 license permits commercial use. Users can run the model with vLLM 0.32.0 and the NeMo Speech plugin, Transformers, or SGLang-Omni 0.2.0. vLLM offers an API compatible with OpenAI.
The model transcribes 40 languages, supports translation between English and 31 of those languages, and identifies speakers in 13, Nvidia said. It can process recordings up to an hour long with as many as eight speakers. Features vary by language. Audio question-answering was trained in English. The model's designed for recorded audio, not livestreams.
Users are advised by Nvidia to review names, changes in speakers, and answers. Hugging Face hosts the model and dataset, along with a demo, fine-tuning guides, and an extension for running the training workflow through an AI coding assistant.
Nvidia released Nemotron 3 Diarization in September, a model built specifically to recognize speakers. It distinguishes as many as eight speakers in live conversations, but transcription required another model, such as Nvidia's Parakeet. Nvidia had previously used the Granary dataset in its multimodal Nemotron 3 Nano Omni. This release adds to Nvidia's open-weight voice models, including Parakeet at 1.1 billion parameters.
Mistral's Voxtral 3 was the latest release this month. ElevenLabs introduced Realtime, designed to provide instant speech for voice agents, while Salesforce put out Launa-Omni. Mistral's Voxtral models handle both transcription and audio understanding. The company recently announced Voxtral 3 for real-time voice agents. These tools target developers building apps for business meetings and medical transcription, two of AI's most established workplace uses.
