Text normalization challenge
RNN Approaches to Text Normalization: A Challenge
In one sentence
Neural sequence-to-sequence models can learn to turn written text ("$3.50", "12/03", "Dr.") into spoken words, and they get almost every token right, but the few they get wrong are the kind no product can ship, so text normalization is posed as an open challenge rather than a solved problem.
Why this paper exists
Every TTS system needs a front-end: the stage that turns raw text into a sequence of words (and then phonemes) that an acoustic model can read. The hard part is text normalization, converting non-standard tokens into words: "$3.50" → "three dollars fifty cents", "1/2" → "one half" or "January second" depending on context, "St." → "Saint" or "Street".
In 2016 production systems did this with large hand-written grammars compiled into finite-state transducers (FSTs: rule machines that rewrite strings), costing years of linguist effort per language. Sproat and Jaitly asked whether a neural network could replace them: they built a dataset by running Google's rule-based normalizer over Wikipedia text, trained RNNs on it, and studied the errors.
The core idea
Treat normalization as translation from written form to spoken form, one token at a time, with the surrounding words as context. Then measure not just accuracy but the type of errors. Their finding: plain words are trivially copied, most numbers and dates are read correctly, and the remaining mistakes are "silly errors": confident readings that change the meaning, such as speaking the wrong number or the wrong unit. A human listener forgives a slightly odd pronunciation; they do not forgive "three hundred" when the text said "thirty".
How it works
- Each token is tagged with a semiotic class: what kind of thing it is (PLAIN word, PUNCT, CARDINAL number, ORDINAL, DATE, MEASURE, MONEY, TIME, ELECTRONIC addresses, LETTERS for acronyms, and so on). The class decides how it should be read.
- The model sees the token to normalize plus a window of words around it (context matters: "2" in "Chapter 2" vs "2 kg").
- An attention-based encoder-decoder RNN reads the input characters and emits the spoken words. Most tokens map to themselves (a special "self" output), which is why overall accuracy looks very high.
- They report accuracy per semiotic class and examine the errors one by one.
- The paper argues for constraining the network with lightweight hand-written grammars ("covering grammars") that list the legal readings of, say, a measure expression, so the network only chooses among sensible options. Check the paper and its 2019 follow-up (Zhang, Sproat et al., "Neural Models of Text Normalization for Speech Applications") for how far this was taken.
- The dataset was released for English and Russian and later became a Kaggle competition run by Google.
Build the intuition
Run this sentence through your head: "Dr. Sharma paid ₹1,50,000 on 12/03 at 5:30 pm for 2 kg of rice at St. Mary's."
- "Dr." → "Doctor" (but "Dr." in an address is "Drive").
- "₹1,50,000" → "one lakh fifty thousand rupees". An English-trained model that has only seen "150,000" will likely say "one hundred fifty thousand", or worse, misparse the comma grouping.
- "12/03" → "twelfth of March" in India, "December third" in the US. No amount of local context in the string fixes this; it needs a locale.
Now imagine a voice agent for a lender reading out an EMI amount. A 99.9%-accurate normalizer that once in a thousand turns reads ₹15,000 as "one lakh fifty thousand" is a compliance incident. That is the paper's point: the error rate is not the right metric; the error type is.
Numbers worth remembering
- Data: Wikipedia text normalized by Google's rule-based system (Kestrel), for English and Russian; check the paper for the exact token counts.
- Overall token accuracy is very high (well above 99%), mostly because the large majority of tokens are plain words copied unchanged.
- The interesting classes (numbers, dates, measures, money) have far lower accuracy than PLAIN, and that is where the "silly errors" live; check the per-class tables in the paper.
Where it shows up today
- Every production TTS still has a normalization stage before the acoustic model, often rules plus a model: the Tacotron line (Tacotron, Tacotron 2) and FastSpeech 2 (FastSpeech 2) assume normalized input; even end-to-end character models such as F5-TTS (F5-TTS) read numbers badly without it.
- LLM-driven voice agents: the LLM's output ("Your EMI is ₹4,999/mo") must be normalized before TTS. A common pattern now is to ask the LLM itself to write numbers out in words, with rules as a safety net.
- Indian-language TTS (Indic TTS (AI4Bharat)) adds code-mixing (Hinglish written in Latin or Devanagari), Indian digit grouping, and multiple scripts.
What it got wrong, and what came next
- The training data was produced by an existing rule system, so the network can at best imitate that system, including its mistakes.
- Pure neural normalization never displaced rules for high-stakes classes. The follow-up work moved toward hybrids: neural models proposing readings, grammars restricting them.
Check yourself
Q1What is a semiotic class, and why does normalization need it?
It is the category of a token (cardinal, ordinal, date, money, measure, plain word and so on). The same characters are read differently depending on class: "2" is "two" as a cardinal and "second" in a date. Predicting or using the class tells the system which reading rules apply.
Q2Why is overall token accuracy a misleading metric here?
Most tokens are ordinary words that are copied unchanged, so a model can score above 99% while still misreading many numbers. The errors that matter are concentrated in a few classes, and some of them change meaning, so you need per-class accuracy and an error taxonomy.
Q3What is a "silly error", and why does it matter more than an awkward reading?
It is a confident output that is fluent but wrong in content, such as the wrong number, unit or date. An awkward reading still conveys the right information; a silly error conveys false information, which is unacceptable in products that speak prices, amounts or medication doses.
Q4What would happen if you fed raw LLM output with "₹1,50,000" straight into an English-trained TTS model?
It may read the digits one by one, apply Western grouping ("one hundred fifty thousand"), skip the ₹ symbol, or mumble. The acoustic model has no reliable notion of the Indian numbering system unless the front-end converts it to "one lakh fifty thousand rupees" first.
When you're back at a laptop
- Write an Indian-English normalizer in Python for: amounts with ₹ and Indian grouping (lakh, crore), dates in DD/MM/YYYY, times, phone numbers read digit by digit, and common abbreviations (Dr., St., Rs., EMI).
- Build a test file of 50 tricky sentences, including Hinglish ones, and run them through your normalizer and through an LLM prompt that asks for "spoken form"; count silly errors for each.
- Feed both outputs to a TTS model (Kokoro or Piper) and listen.