articles / / 6 min read
Misheard: matching Indian names by how they sound
Edit distance thinks Mohan and Rohan are close and Barmer and बाड़मेर are far apart. misheard fixes both, across Devanagari, Tamil and Roman script.
A caller tells a voice agent they live in Barmer. The speech-to-text engine writes Baadmer. The district list says Barmer, and the customer record says बाड़मेर. A person would see that all three are the same place in a second. String matching doesn't.
I ran into this building Vaani, a multilingual voice agent that has to get customers' names and districts right before it can verify who it's talking to. The matching engine I wrote for it turned out to be the most reusable part, so I pulled it out into a library: misheard. It's pure Python with no dependencies, and version 0.1 is on PyPI.
pip install misheard
Edit distance is wrong in both directions
The usual tool for fuzzy matching is edit distance: count the letters you'd have to change to turn one string into the other. For Indian names it makes two kinds of mistakes.
It makes the same name look different. Barmer and Baadmer differ by two edits, but they're one place. The long vowel is written aa, and the retroflex flap ड़ is heard and written as either r or d. Those are spelling differences, not different words.
It makes different names look the same. Mohan and Rohan differ by one letter, so edit distance calls them close. But m and r sound nothing alike, and these are two different people. A voice agent that confuses them has verified the wrong customer.
English phonetic codes like Soundex and Metaphone don't help much. They follow English spelling rules, and they know nothing about retroflex consonants, aspiration, or the silent a in Hindi. And neither approach can compare a Roman spelling with a name written in Devanagari or Tamil at all.
What misheard does instead
from misheard import distance, best_match, to_roman
to_roman("बाड़मेर") # 'baaṛmer'
to_roman("முருகன்") # 'murugan'
distance("Barmer", "बाड़मेर") # 0.4 same place
distance("Barmer", "Baadmer") # 0.6 same place, different spelling
distance("Mohan", "Rohan") # 1.7 different people
Lower means closer, and 0 means the same sounds. It works in five steps.
1. Transliterate to one phonetic form. Devanagari and Tamil are converted into a single phonetic Roman spelling. This is more than a lookup table. Hindi drops the inherent a at the end of words (मोहन is mohan, not mohana). Tamil voices stops between vowels (முருகன் is murugan, even though Tamil writes the k and g sounds with one letter). Retroflex sounds keep their marks (ṭ ḍ ṇ ḷ ṛ) so that information isn't lost.
2. Tokenize into sounds. Common English-style spellings are folded first (x becomes ks, doubled letters collapse), then the word is split into sounds rather than letters. Bh is one sound, not two.
3. Score substitutions by how the sounds are made. Each consonant is described by its place and manner of articulation, voicing and aspiration. Each vowel is described by its height, backness and length. Swapping two similar sounds is cheap, and swapping unrelated ones is expensive. That's why t/ṭ costs little and m/r costs a lot.
4. Override with known confusions. Some sound pairs are confused far more often than their phonetic features suggest. These go in a small CSV, and every row says why it's there:
a,b,cost,why
r,ṛ,0.15,"retroflex flap ड़ is heard and written as r (Baadmer/Barmer)"
v,w,0.15,"no phonemic v/w contrast in most Indian languages"
ph,f,0.15,"फ is often pronounced f"
k,g,0.25,"Tamil uses one letter for both"
5. Compare whole names the way people say them. Titles and honorifics (Shri, Smt, Dr, ji, sahab) are ignored. A joined honorific like Rameshbhai costs 0.1 to drop. An initial matches a word with the same first letter (R. Kumar vs Ramesh Kumar is 0.3). Words can come in any order for a small penalty, because Kumar Ramesh Singh and Ramesh Kumar Singh are the same person.
Every distance can explain itself
A number on its own is hard to trust, so misheard can show its working. From the command line:
$ misheard Barmer बाड़मेर
Barmer ↔ बाड़मेर (baaṛmer) distance 0.4
b = b
a → aa 0.25 vowel length
r → ṛ 0.15 retroflex flap ड़ is heard and written as r (Baadmer/Barmer)
m = m
e = e
r = r
The same breakdown is available in Python through explain(a, b). It turned out to be the most useful debugging tool I had while building Vaani. When a match goes wrong, you can see which step caused it instead of guessing.
For lookups against a list, a Matcher prepares the references once, and keys group aliases like Bangalore and Bengaluru under one result:
from misheard import Matcher
people = [("C1", "Rohan Verma"), ("C2", "Mohan Verma"), ("C3", "Sohan Verma")]
Matcher(people).best_match("Rohun Varma")
# [Match(key='C1', text='Rohan Verma', distance=1.7),
# Match(key='C3', text='Sohan Verma', distance=2.85),
# Match(key='C2', text='Mohan Verma', distance=3.4)]
Regions differ, so the costs are yours to change
The default confusion table is tuned for India as a whole, but accents aren't uniform. In Bihar, Bengal and Odisha, v is often said as b, so Vinod comes through as Binod. Making that pair cheap everywhere would be a mistake, though, because Binod is also a real name. A customer list containing both a Vinod and a Binod needs them kept apart.
So the table is a value you can adjust without touching the library:
from misheard import Confusions, Matcher
bihar = Confusions.default().merged({("v", "b"): (0.1, "b for v in Bihar")})
strict = Confusions.default().without(("v", "b"))
Matcher(people, confusions=bihar).best_match("Binod Kumar")
misheard also doesn't decide whether a match is good enough, because that depends on what a wrong answer costs you. In Vaani, the agent accepted a match below about 0.8 if it was at least 1.0 ahead of the next candidate, read back anything below 1.8 ("Did you say Barmer?"), and asked again otherwise. Those thresholds belong in your application, tuned on your data. I wrote about how Vaani uses them, along with dates and numbers, in Vaani: a voice agent that gets names, dates and numbers right.
The benchmark, and why you shouldn't trust it yet
The benchmark has 782 items: names matched against 300 synthetic people, and places matched against 745 Indian districts plus aliases. Top-1 accuracy:
| Source | Kind | Levenshtein | Metaphone | misheard |
|---|---|---|---|---|
| handwritten | name | 96.8% | 96.8% | 100.0% |
| handwritten | place | 92.0% | 74.0% | 100.0% |
| native script | name | 97.6% | 97.6% | 100.0% |
| rule-generated | name | 99.6% | 96.3% | 100.0% |
| rule-generated | place | 97.6% | 90.0% | 100.0% |
100% everywhere should make you suspicious, and it makes me suspicious too. None of these items are real speech-to-text output. The rule-generated variants come from the same confusions the engine models, and I wrote the handwritten ones myself. The benchmark mainly proves that misheard does what I designed it to do. It doesn't yet prove that the design matches how real callers and real STT engines get names wrong.
There are known gaps as well. Patil and Patel score 0.35, close enough to be accepted, and that was the one false accept in Vaani's own evaluation. Speed is behind too: about 5–6 ms per lookup against 780 places, compared with well under a millisecond for Levenshtein.
What's next
The roadmap is in PLAN.md, and the next phase matters more than any feature:
- An honest benchmark (0.2): real spoken names and places in Hindi, Marathi, Tamil and Indian English, run through actual STT services. It will report precision at a threshold instead of just top-1, include names that shouldn't match anything, and compare against stronger baselines like Jaro-Winkler and IndicXlit plus Levenshtein.
- Speed (0.3): a Rust kernel for the weighted edit distance, with the pure-Python version kept as the reference it's tested against.
- Learned costs (0.5): fit the confusion costs from confirmed and corrected read-backs, with regional and per-STT-engine profiles. It stays a readable table, not a neural model.
- More scripts: Telugu, Kannada, Bengali, Gujarati, Gurmukhi and Malayalam. Unsupported scripts raise an error today instead of being silently dropped.
Try it
pip install misheard
misheard Lakshmi லட்சுமி
If you're building anything that hears or types Indian names, such as voice agents, KYC, call-centre tools, or deduplicating records, I'd love to hear where it gets things wrong. Real failure cases are exactly what the next benchmark needs. Issues are open on GitHub.