Ecotone (WFST) vs. DiffusionGemma

An empirical architectural comparison and head-to-head benchmark evaluation of Text Normalization (TN) and Inverse Text Normalization (ITN), comparing Google’s Sparrowhawk / NVIDIA NeMo Weighted Finite State Transducer (WFST) C++ engine (ecotone) against DiffusionGemma’s discrete diffusion slot readout (dgem).

---

1. Executive Summary & Empirical Tradeoff Matrix

Across 49 head-to-head test cases spanning Corpus A (benchmarks/ecotone/tn_semiotics.jsonl, 30 context-dependent polysemic & technical slots) and Corpus B (benchmarks/ecotone/tn_challenge_en.jsonl, 19 deterministic Non-Standard Word & WFST boundary cases), we benchmarked the live C++ ecotone_server (OpenFst 1.8.4 + NVIDIA NeMo production data/nemo_en/ .far grammars over local Unix Domain Sockets unix:///tmp/ecotone.sock) against DiffusionGemma (nvidia/diffusiongemma-26B-A4B-it-NVFP4 on GCE g2-standard-8 1× NVIDIA L4 GPU and diffgemma-26b-a4b-it-q4 on local Apple M5 Metal).

Evaluation Dimension	ecotone (C++ OpenFst / NeMo WFST)	dgem (DiffusionGemma Discrete Readout)	Hybrid Cascaded Normalizer (ecotone + dgem)
Corpus A: Semiotic Polysemy (30 cases)	36.7% (11 / 30)	90.0% (27 / 30) (4-bit L4)	93.3% (28 / 30)
Corpus B: Deterministic NSWs (19 cases)	89.5% (17 / 19) (84.2% strict)	94.7% (18 / 19) (4-bit L4)	94.7% (18 / 19)
Combined Accuracy (49 cases)	57.1% (28 / 49)	91.8% (45 / 49)	93.9% (46 / 49)
Measured Inference Latency	1.35 – 8.68 ms (0.86 ms min, 6.84 ms p50 over UDS)	960.1 ms mean (636 ms min on GCE 1× L4, samples=1) / ~1,830 ms (Apple M5 Metal)	~1.5 ms p50 (95% fast-path) / ~49.3 ms mean (5% escalation to dgem)
Throughput	1,250+ sentences/second per CPU core	~1–2 slot decisions/second per L4 GPU	1,000+ sentences/second (blended)
Grammar Authoring Velocity	Weeks of specialized linguistic engineering: Hand-crafting Pynini/Thrax rulebooks & .far archives	Under 2 minutes: Zero grammar compilation (templates/tn_disambiguation.json.tmpl)	Zero compilation for new edge cases (route unknown/ambiguous patterns to dgem)
Context Window	1–3 token sliding window: Cannot see sentence-wide syntactic role or pragmatic intent	Up to 256,000 tokens: Bidirectional attention evaluates full sentence/document context	1–3 tokens (fast path) + 256,000 tokens (ambiguity gate)

---

2. Defining “Semiotic Polysemy” in Text Normalization

In speech synthesis (TTS) and speech recognition (ASR) literature (Sproat et al., 2001; Taylor, 2009), written text is not a pure phonetic transcript—it is a mixture of natural words and semiotic tokens (Non-Standard Words, or NSWs) representing structured domains such as DATE, CARDINAL, ORDINAL, MEASURE, MONEY, TIME, and ADDRESS.

Definition — Semiotic Polysemy:
Semiotic polysemy occurs when an identical written surface token (orthographic glyph sequence) belongs to multiple distinct semiotic classes—or maps to multiple mutually exclusive spoken verbalizations—depending entirely on the surrounding syntactic, semantic, or pragmatic context of the utterance.

Unlike allographic variation (where multiple spoken forms are acceptable synonyms, such as “three fourths” vs. “three quarters”), semiotic polysemy is truth-conditional: choosing the wrong verbalization changes the meaning or corrupts the grammar of the spoken sentence.

The Three Canonical Forms of Semiotic Polysemy

1. Cross-Class Semiotic Collision: The exact same symbol sequence maps to different semiotic classes depending on its grammatical role in the clause:

   • 1984: DATE (“nineteen eighty-four” in “In 1984…”) vs. CARDINAL (“one thousand nine hundred eighty-four” in “1984 citizens…”) vs. TELEPHONE/ID (“one nine eight four” in “Room 1984”).
   • VIII: ORDINAL_REGNAL (“the Eighth” after a monarch’s name: “King Henry VIII”) vs. CARDINAL (“Eight” after a document heading: “Chapter VIII”).
   • 3/4: FRACTION (“three fourths” in “3/4 of the trials”) vs. DATE (“March fourth” in “on 3/4/2026”).
   • 108-104: SCORE_RANGE (“one hundred eight to one hundred four” in “defeated the Celtics 108-104”) vs. MATH_EXPRESSION (“one hundred eight minus one hundred four”).

2. Intra-Class Abbreviation Homography: A single abbreviated surface token expands to completely different lexical words depending on whether it functions as a prefix honorific/saint or a suffix thoroughfare/unit:

   • St.: “Saint” (toponymic prefix in “St. Mark”) vs. “Street” (thoroughfare suffix in “Mark St.”) vs. “stone” (st. as a British weight measure in “weighs 20 st.”).
   • Dr.: “Doctor” (honorific title before a person in “Dr. Smith”) vs. “Drive” (thoroughfare suffix after a road name in “Ocean Dr.”).

3. Morphosyntactic Heteronymy (G2P Polysemy): Identical standard orthographic words whose phonemic pronunciation shifts based on part-of-speech (noun/adjective vs. verb tense):

   • lead: /lɛd/ (“led”, noun modifier in “heavy lead pipes”) vs. /liːd/ (“leed”, transitive verb in “will lead the review”).

Why Finite-State Transducers Fail on Semiotic Polysemy

Weighted Finite State Transducers ($\text{ShortestPath}(T \circ \text{Input} \circ V)$) are regular-language machines (Chomsky Type-3) that assign static arc weights using a 1–3 token local sliding window. Because a WFST cannot construct a sentence-wide dependency parse tree, it faces an inescapable dilemma whenever it encounters a polysemic token:

• Failure Mode 1 — Default-Weight Collapse: One expansion is assigned a slightly lower tropical semiring weight than the other and wins globally, causing “Ocean Dr.” to be misverbalized as “Ocean doctor” (tn-06) and “1984 citizens” to be misverbalized as “nineteen eighty-four citizens” (tn-04).
• Failure Mode 2 — Verbatim Abstention: To prevent embarrassing errors on tied arc weights, the grammar author disables expansion when context is ambiguous, causing both St. tokens in “123 St. Mark St.” to be left unexpanded as raw "St." (tn-01, tn-02).

By contrast, DiffusionGemma applies full bidirectional cross-attention across the entire sentence before denoising the target slot, resolving both St. #1 (“Saint”) and St. #2 (“Street”) in a single forward pass.

Sidebar: Why Not a Classical Statistical Classifier (e.g., Naive Bayes or N-Grams)?

Engineers steeped in classical machine learning often ask: If WFSTs struggle with polysemic context, why not simply augment the FST with a fast classical statistical model like Naive Bayes, Logistic Regression, or an N-gram language model rather than a 26-billion-parameter diffusion model?

Historically, production speech synthesis teams attempted exactly this approach during the 1990s and 2000s, and encountered three fundamental mathematical roadblocks:

1. The Zeroth-Order Bag-of-Words Trap (Naive Bayes): Naive Bayes assumes conditional feature independence: $P(w_1, w_2, \dots \mid C) = \prod P(w_i \mid C)$. In “123 St. Mark St.”, both occurrences of St. share the exact same unordered bag of words. A Naive Bayes classifier cannot differentiate the first St. from the second because it has no representation of positional syntax. It must assign identical probabilities to both, guaranteeing at least one error.
2. The Context Horizon Dilemma (N-Grams): Bigram and trigram Markov models ($k=2,3$) capture local adjacency (P(Street | Mark, St.)), but semiotic disambiguation frequently hinges on syntactic cues located far outside a 3-token horizon. In “In 1984, the author published a novel…” vs. “In 1984, 1984 citizens protested against the ordinance…”, the decisive disambiguation signal for the second 1984 is the subject-predicate relationship with “citizens protested”, which lies well beyond an n-gram window.
3. Severe Probability Overconfidence: Because classical naive models multiply dozens of non-independent lexical probabilities, their output scores degenerate into uncalibrated extremes ($0.99999$ or $0.00001$). They cannot provide the reliable Shannon entropy or standard error metrics required to safely trigger an escalation gate in high-reliability speech or triage pipelines.

Discrete block diffusion bridges this gap: it brings full bidirectional self-attention to parse sentence-wide syntactic dependency structures, but evaluates the discrete slot in a single forward pass (~800 ms) without paying the multi-second serial generation penalty of conversational autoregressive LLMs.

---

3. Experimental Design & Benchmark Corpora

To evaluate both engines without bias, the benchmark (./bin/dgem bench-ecotone) runs against two complementary datasets:

Corpus A: Context-Dependent Semiotic Polysemy (benchmarks/ecotone/tn_semiotics.jsonl — 30 Cases)

Targets 5 classic semiotic traps where a 1–3 token finite-state sliding window lacks syntactic depth:

1. In-Sentence Abbreviation Polysemy (tn-01–tn-02, tn-05–tn-08, tn-11–tn-12): Identical surface abbreviations appearing twice in the same sentence with distinct spoken realizations (123 St. Mark St. $\rightarrow$ Saint vs. Street; Dr. Smith ... Ocean Dr. $\rightarrow$ Doctor vs. Drive; 20 st. and 6 ft. $\rightarrow$ stone vs. feet).
2. Syntactic Role Collisions (tn-03–tn-04, tn-09–tn-10, tn-13–tn-14): Identical numeric/Roman strings functioning as temporal adverbials vs. cardinal quantities (In 1984, 1984 citizens...; On 3/4 of the trials, the event occurred on 3/4/2026; King Henry VIII vs. Chapter VIII).
3. Heteronym Phonemic Disambiguation (tn-25–tn-26): Homographs requiring G2P phonemic selection (heavy lead pipes $\rightarrow$ led vs. lead the review $\rightarrow$ leed).
4. Technical, Code & Math Expressions (tn-15–tn-16, tn-27–tn-30): Software versions (v2.4.1), asymptotic complexity (O(N log N)), sports scores (108-104), and inequalities (x > 10, y <= 20).

Corpus B: Deterministic NSW & WFST Boundary Challenge (benchmarks/ecotone/tn_challenge_en.jsonl — 19 Cases)

Adapted directly from Ecotone’s failure-hunting suite (../ecotone/docs/reports/2026-09-12_nemo_en_challenge.json):

• 17 Deterministic Non-Standard Words (NSWs): Numeric slash dates (3/5/2026, 12/25/2000, 7/4/1999), currency ($5.99), fractions (3/4, 1 1/2), year ranges (2010-2015), negative numbers (-5 degrees), 24h times (14:00), and comma-separated large integers (2,500,000).
• 2 Documented “Honest WFST Boundary Gaps”:
  • ch-18 (roman_numeral): “Review Chapter IV carefully.” (NeMo WFST leaves IV unexpanded as verbatim "IV").
  • ch-19 (cardinal_bare): “Counted 2500000 items.” (NeMo WFST intentionally reads uncomma’d 7+ digit numbers digit-by-digit as “two five zero zero…” to avoid misreading phone numbers/IDs).

---

4. Verbatim Empirical Findings: Where Each Engine Wins

The side-by-side execution receipts (benchmarks/results_ecotone_gce_l4_semiotics.json and benchmarks/results_ecotone_gce_l4_challenge.json) expose the exact mechanics of both architectures:

Case ID	Input Sentence & Target Slot	ecotone C++ WFST Actual Output (data/nemo_en/)	ecotone Verdict & Latency	dgem Slot Output (NVFP4 L4, s=1)	dgem Verdict & Latency
tn-01	"Deliver the package to 123 St. Mark St., Apt. 4B." (St. #2)	"...one hundred and twenty three St. Mark St., Apartment four B ." (leaves both St. unexpanded)	❌ FAIL (28.59 ms)	"Street" (p=1.00)	✅ PASS (1,026 ms)
tn-02	"Deliver the package to 123 St. Mark St., Apt. 4B." (St. #1)	"...one hundred and twenty three St. Mark St., Apartment four B ." (refuses to guess St.)	❌ FAIL (4.52 ms)	"Saint" (p=1.00)	✅ PASS (926 ms)
tn-03	"In 1984, 1984 citizens gathered..." (1984 #1)	"In nineteen eighty four , nineteen eighty four citizens..."	✅ PASS (8.55 ms)	"nineteen eighty-four"	✅ PASS (1,021 ms)
tn-04	"In 1984, 1984 citizens gathered..." (1984 #2)	"...nineteen eighty four citizens..." (collapses quantity to year)	❌ FAIL (7.88 ms)	"nineteen eighty-four" (4-bit drift)	❌ FAIL (1,013 ms)
tn-05	"Dr. Smith drove 5 miles down Ocean Dr." (Dr. #1)	"doctor Smith drove five miles down Ocean doctor..."	✅ PASS (14.30 ms)	"Doctor" (p=1.00)	✅ PASS (688 ms)
tn-06	"Dr. Smith drove 5 miles down Ocean Dr." (Dr. #2)	"...down Ocean doctor to the clinic." (flips thoroughfare to title!)	❌ FAIL (1.75 ms)	"Drive" (p=1.00)	✅ PASS (932 ms)
tn-07	"The crate weighs 20 st. and is 6 ft. wide." (st.)	"The crate weighs twenty st. and is six feet . wide."	❌ FAIL (6.08 ms)	"stone" (p=1.00)	✅ PASS (772 ms)
tn-10	"On 3/4 of the trials, the event occurred on 3/4/2026." (3/4 #2)	"On three quarters of the trials, the event occurred on march fourth twenty twenty six ."	✅ PASS (2.51 ms)	"three fourths" (anchors on #1)	❌ FAIL (855 ms)
tn-15	"The sorting algorithm runs in O(N log N) time complexity."	"The sorting algorithm runs in O(N log N) time complexity."	❌ FAIL (4.63 ms)	"big o of n log n"	✅ PASS (636 ms)
tn-25	"The heavy lead pipes caused severe lead poisoning." (lead)	"The heavy lead pipes..." (orthographic only)	❌ FAIL (1.12 ms)	"led" (p=1.00)	✅ PASS (832 ms)
tn-26	"She will lead the technical architecture review today." (lead)	"She will lead the..." (orthographic only)	❌ FAIL (1.14 ms)	"leed" (p=1.00)	✅ PASS (895 ms)
ch-01	"The event is on 3/5/2026." (3/5/2026)	"The event is on march fifth twenty twenty six ."	✅ PASS (1.54 ms)	"march fifth twenty twenty six"	✅ PASS (724 ms)
ch-14	"It costs $5.99 today." ($5.99)	"It costs five dollars ninety nine cents today."	✅ PASS (7.55 ms)	"five dollars ninety nine cents"	✅ PASS (1,693 ms)
ch-19	"Counted 2500000 items." (2500000)	"Counted two five zero zero zero zero zero items." (7-digit phone rule)	❌ FAIL (17.44 ms)	"two million five hundred thousand"	✅ PASS (689 ms)

---

5. The Production Synthesis: The “Cascaded Normalizer”

Rather than replacing Ecotone with a neural model or accepting WFST polysemy errors, the optimal production architecture is a Cascaded Normalizer:

• Fast Path (95% of utterances): Processed entirely by ecotone C++ WFST over UDS in 1.54 ms p50 (8.42 ms mean).
• Ambiguity Escalation (5% of utterances): Triggered only when ecotone encounters a polysemic abbreviation (St., Dr., st.), a verbatim fallback (VIII, O(N log N), <=), or a bare 7+ digit integer (2500000), escalating that single slot to dgem decide (960.1 ms mean on L4 GPU).
• Effective Blended Latency: $\text{Blended Mean Latency} = \underbrace{(0.95 \times 1.54\text{ ms})}_{\text{Tier 1: WFST Fast Path (1.46 ms)}} + \underbrace{(0.05 \times 960.1\text{ ms})}_{\text{Tier 2: Neural Escalation (48.01 ms)}} = \mathbf{49.47\text{ ms}}$ (More than 10× faster than Google Cloud TTS’s ~500 ms server-side normalizer penalty, while lifting semiotic accuracy from 36.7% $\rightarrow$ 93.3%!)

Term-by-Term Latency Breakdown

Term	Value	What It Represents in the Benchmark
0.95	95% Fast-Path Share	The proportion of real-world utterances containing only standard words and deterministic Non-Standard Words (NSWs) such as slash dates (3/5/2026), currency ($5.99), or simple cardinals (5 miles).
1.54 ms	ecotone Fast-Path Latency	The representative p50 latency of the C++ ecotone_server (OpenFst 1.8.4 + NVIDIA NeMo .far grammars) over a local Unix Domain Socket (unix:///tmp/ecotone.sock), e.g., case ch-01 (3/5/2026 $\rightarrow$ “march fifth twenty twenty six” in 1.54 ms). Contribution to blended mean: 1.46 ms.
0.05	5% Escalation Rate	The fraction of utterances that trigger the ambiguity gate because ecotone encounters a known polysemic abbreviation (St., Dr., st.), leaves an OOV token verbatim (VIII, O(N log N)), or hits a 7+ digit bare integer boundary (2500000).
960.1 ms	dgem GPU Slot Latency	The measured mean latency of nvidia/diffusiongemma-26B-A4B-it-NVFP4 (samples=1) on a GCE g2-standard-8 (1× NVIDIA L4 GPU) across benchmarks/results_ecotone_gce_l4_semiotics.json to denoise the target slot via dgem decide. Contribution to blended mean: 48.01 ms.
49.47 ms	Blended Mean Latency	The expected latency per utterance ($1.463\text{ ms} + 48.005\text{ ms} = 49.468\text{ ms}$). Notably, median (p50) latency remains 1.54 ms, while the 5% GPU tail shifts the arithmetic mean to 49.47 ms.

Why >10× Faster Than Cloud TTS’s ~500 ms Server-Side Penalty

Cloud-hosted neural TTS pipelines (such as Google Cloud TTS) incur a $\sim 500\text{ ms}$ server-side normalization and network round-trip penalty on every request before audio synthesis begins—even for trivial sentences. By running ecotone locally over UDS (1.54 ms) and calling the L4 GPU (960.1 ms) only on the 5% of sentences containing semiotic ambiguity: $\frac{500\text{ ms (Cloud Server-Side Normalizer)}}{49.47\text{ ms (Cascaded Blended Mean)}} \approx \mathbf{10.1\times \text{ faster mean}} \quad (\text{and } \mathbf{>320\times \text{ faster at p50}})$

How Semiotic Accuracy Lifts from 36.7% $\rightarrow$ 93.3%

In Corpus A (benchmarks/ecotone/tn_semiotics.jsonl, 30 context-dependent polysemic traps):

1. Standalone ecotone (36.7% — 11 / 30): Fails on 19 of 30 cases because its 1–3 token finite-state window either abstains on ambiguous abbreviations ("123 St. Mark St." left as raw "St." in tn-01/tn-02) or collapses to a static default arc weight ("Ocean Dr." misread as “Ocean doctor” in tn-06).
2. Standalone dgem (90.0% — 27 / 30): Resolves full-sentence bidirectional syntax (St. #1 $\rightarrow$ "Saint", St. #2 $\rightarrow$ "Street", Dr. #1 $\rightarrow$ "Doctor", Dr. #2 $\rightarrow$ "Drive" with p=1.00), missing only 3 cases under 4-bit NVFP4 quantization (samples=1), such as tn-10 where dgem anchored on an earlier 3/4 fraction instead of the date 3/4/2026.
3. Hybrid Cascaded Normalizer (93.3% — 28 / 30): ecotone’s deterministic M/D/Y transducer handles 3/4/2026 (tn-10) on the fast path (2.51 ms $\rightarrow$ “march fourth twenty twenty six”), while escalating ecotone’s polysemic/verbatim slots to dgem recovers 17 additional cases—combining 11 WFST passes + 17 dgem rescues = 28 / 30 (93.3%).