Devnagri Black Bird Records Lowest Error Across 15 Indian Languages on Voice of India Benchmark

Devnagri's new ASR model records the lowest average error across 15 Indian languages, ahead of Sarvam Saaras V3 and Google's Gemini 3 Pro

NOIDA, UP, INDIA, September 30, 2026 /EINPresswire.com/ -- Devnagri AI today announced that its new automatic speech recognition (ASR) model, Devnagri Black Bird, leads the Voice of India benchmark, the independent Indian ASR test set built by AI4Bharat at IIT Madras with Josh Talks. Devnagri Black Bird recorded an average Open Indic Word Error Rate (OI-WER) of 8.7, the lowest of every system evaluated.

At 8.7, Devnagri Black Bird's average error is less than half of Google's Gemini 3 Pro (21.1). As of Devnagri's September 2026 evaluation, it is about 19 percent lower than Sarvam Saaras V3 (10.7), about 59 percent lower than Gemini 3 Pro, and about 62 percent lower than Gemini 3 Flash (23.1). It also cuts errors by about 25 percent versus Devnagri's previous model, Shivaani ASR.

Dated April 21, 2026, Voice of India was built to test speech technology on how India actually talks. It draws on 536 hours of unscripted, real-world telephonic conversations across 15 Indian languages, spanning 306,230 utterances and 36,691 native speakers, with cohorts balanced for gender and age. Unlike clean, scripted test sets, it captures the noise, regional accents, and code-mixing of everyday phone calls, and it accepts valid spelling and transliteration variants rather than penalizing natural speech. This is the environment Devnagri Black Bird was built for: an Indian-language speech intelligence layer developed around code-mixed speech, regional accents, noisy telephony, and the deployment constraints of Indian contact centres and voice-agent workloads.

Devnagri evaluated all nine systems the same way, by calling each one's production API and scoring the output against the same references under identical default settings. The test set is closed to prevent overfitting, and model versions and logs were retained so the results can be verified. The other players' results reported here were measured by Devnagri using publicly accessible production APIs, and should not be interpreted as vendor-published results. Devnagri Black Bird leads on all 15 languages, including lower-resource ones such as Bhojpuri and Maithili that global systems handle poorly, so the lead is not limited to high-resource languages like Hindi.

The market context is what makes this matter for the product. Around 90 percent of India's digital users prefer to transact in a language other than English, and in BFSI more than 70 percent of the digital population is non-English. As enterprises move onboarding, collections, servicing, and support to voice and phone channels, and as growth shifts to Tier 2 and Tier 3 India, speech-recognition accuracy in regional languages becomes the constraint on what can actually be automated. In enterprise voice AI, ASR is the first intelligence layer. If the system mishears an amount, a name, a date, or an intent, that error flows into every downstream step.

That is why accuracy on real calls matters most in BFSI, government citizen services, and digital commerce, where high-value conversations happen by phone, in a regional language, often over a noisy line. In banking, financial services, and insurance, correctly capturing names, dates, amounts, and identity details improves customer onboarding and KYC, payment collections, renewal reminders, support calls, and grievance handling. Fewer transcription errors mean fewer failed workflows, lower compliance risk, and less repeated effort, while diarization, timestamps, and correctly rendered numbers keep every interaction auditable. In government services, the same capability lets helplines and citizen voicebots serve people in their mother tongue. In digital commerce, it powers cash-on-delivery confirmation, delivery and returns support in regional languages, reducing failed deliveries and support load as sales shift to non-metro India.

By leading an independent, real-world benchmark, Devnagri’s new ASR model raises the bar for what Indian enterprises should expect from speech AI - live calls in the languages customers actually speak.

"When a bank can onboard a customer, run a collections call, or resolve a grievance accurately in the customer's own language, that is a larger market it can reach and a far better experience for the person on the line. Our latest ASR model, Black Bird, moves the industry forward because we build for the real conditions of India, not for the lab."
— Nakul Kundra, Co-founder, Devnagri AI

"Leading an independent, real-world telephonic benchmark is genuinely hard. Indian speech is messy: people switch languages mid-sentence, calls are noisy, spellings vary. Staying accurate there, not just on clean audio, is where the real engineering is, and we are improving our ASR model, Black Bird, continuously."
— Himanshu Sharma, Co-founder, Devnagri AI

About Devnagri AI

Devnagri AI builds language infrastructure that helps regulated enterprises standardize multilingual communication across customer journeys, workflows, documents, and voice, in 40+ languages including 22 scheduled Indian languages. Trusted by leading organizations in banking, financial services, insurance, government, retail, and digital commerce, Devnagri delivers governed language infrastructure built for enterprise scale.

Priyanka Harshvardhan
Devnagri AI Private Limited
+91 96505 00268
email us here
Visit us on social media:
LinkedIn
Instagram
Facebook
YouTube
X