Voice Synthesis

Text-to-speech output sounds wrong when the synthesis layer applies a default, one-size-fits-all rendering to text that actually needs domain-specific pronunciation, content-appropriate emotion, or a stable persona, and none of those needs are encoded as explicit metadata the TTS engine can act on. Brand names, acronyms, and heteronyms fall outside a general grapheme-to-phoneme model’s training distribution; emotional register and disfluency need to match message content and persona rather than a single fixed voice style; and markup, waveform generation, and voice selection all have independent failure modes that surface as audible artifacts, mispronunciation, or an inconsistent-sounding agent.

Key Takeaways

• 8 patterns split into three clusters: pronunciation accuracy (brand terms, general pronunciation, prosody), synthesis mechanics (audio artifacts, SSML markup), and persona/affect calibration (emotional tone, disfluency, voice consistency).
• Custom pronunciation lexicons cut name/brand mispronunciation errors by roughly 80%, and domain-term accuracy without hints runs only 50-70% — the single highest-leverage fix in voice synthesis is lexicon coverage, not a better base TTS model.
• SSML tag processing succeeds only 70-85% of the time across engines, and 5-10% of failures result in raw markup being spoken aloud verbatim — a jarring, immediately noticeable defect distinct from subtler mispronunciation.
• Emotional appropriateness in production TTS runs only 50-70%, and inappropriate tone is linked to a 25% reduction in user trust, showing that affect mismatch is as consequential as outright mispronunciation.

Scope

• Pronunciation Accuracy — brand-term-mispronunciation (GitHub), pronunciation-errors (GitHub), prosody-mismatch (GitHub). Grouped because all three concern the TTS engine’s default grapheme-to-phoneme and rhythm rules producing an incorrect or unnatural rendering of otherwise-correct text — wrong phonemes for brand/domain terms and names, or wrong stress/intonation/pacing for sentence-level meaning.
• Synthesis Mechanics — audio-artifact-generation (GitHub), ssml-markup-failures (GitHub). Grouped because both are pipeline/engine-level defects (waveform discontinuities, markup parsing failures) rather than linguistic mispronunciation — the text-to-phoneme mapping can be correct while the audio production or markup interpretation still fails.
• Persona and Affect Calibration — emotional-tone-mismatch (GitHub), disfluency-calibration-failures (GitHub), voice-consistency-issues (GitHub). Grouped because all three concern whether the voice’s emotional register, natural hesitation, and identity stay matched to persona and context across a conversation, rather than any single utterance’s phonetic correctness.

When Voice Synthesis Matters

• The agent must speak brand names, acronyms, technical jargon, or names not in a general TTS lexicon, where mispronunciation directly damages brand perception or trust
• Message content spans emotionally distinct categories — fraud alerts, approvals, denials, appointment reminders — where a mismatched tone (cheerful fraud alert, flat congratulations) is itself a user-facing failure, not just an accuracy one
• The deployment uses SSML markup, multiple languages, or load-balanced TTS infrastructure, where markup-processing failures or voice-selection drift can silently break the intended output

Cross-Pattern Insight

The dominant mitigation across all 8 patterns is making an implicit rendering decision explicit and auditable: custom lexicons and SSML phoneme tags make pronunciation an authored decision instead of a model guess; emotion tags attached to message templates at authoring time make tone a deliberate content decision instead of a runtime inference; and a single canonical voice-persona specification that every code path must reference eliminates the “different voice for errors” class of inconsistency. In every pattern, the failure mode is the TTS engine silently defaulting — to its trained pronunciation rules, its uniform prosody, or whatever voice a given code path happens to select — and the fix is removing that default by supplying explicit metadata upstream of synthesis.

Frequently Asked Questions

What’s the difference between brand-term mispronunciation and general pronunciation errors?

Brand-term-mispronunciation (GitHub) focuses specifically on brand names, acronyms, and version/number formats unique to a product or company. Pronunciation-errors (GitHub) covers the broader class — personal names, heteronyms (read/read), foreign words — that any TTS deployment encounters regardless of brand. Both are fixed the same way: custom lexicons and SSML phoneme overrides checked before default synthesis.

What causes SSML markup to sometimes get read aloud instead of applied?

SSML-markup-failures (GitHub) attributes literal-tag-spoken failures to malformed tags, unsupported engine features, or encoding/escaping issues in dynamically inserted content — the pattern reports 5-10% of SSML usage incidents result in literal tags being spoken, and recommends pre-send validation against an engine-specific capability matrix rather than assuming one SSML dialect works everywhere.

Can a single TTS voice handle both good news and bad news appropriately?

Not without explicit emotion tagging. Emotional-tone-mismatch (GitHub) documents cheerful delivery of fraud alerts and flat delivery of approvals as a common failure, with emotional appropriateness in production running only 50-70% — the fix is classifying each message’s required emotional register at template-authoring time, not relying on the TTS engine to infer it from content alone.

What can cause a voice agent’s voice to change mid-conversation?

Usually unintentional infrastructure behavior: different TTS instances behind a load balancer, a language switch without matched cross-language persona settings, or a different voice hardcoded for error messages versus success messages. Voice-consistency-issues (GitHub) reports 80%+ of users detect voice changes and 60% of those report confusion, recommending session-pinned TTS routing and a centralized voice-persona configuration all code paths must use.

Patterns

Pattern	Mechanism
Audio Artifact Generation (GitHub)	Clicks, pops, and distortion from segment-boundary discontinuities or low-bitrate encoding
Brand Term Mispronunciation (GitHub)	Brand names, acronyms, and version numbers misread by default grapheme-to-phoneme rules
Disfluency Calibration Failures (GitHub)	Filler words and hesitations mismatched to persona or context (too casual for fraud alerts, too robotic for casual sales)
Emotional Tone Mismatch (GitHub)	Voice emotion doesn’t match message content, e.g. cheerful delivery of bad news
Pronunciation Errors (GitHub)	Names, acronyms, foreign words, and heteronyms mispronounced absent a custom lexicon
Prosody Mismatch (GitHub)	Flat intonation, missing emphasis, and wrong pacing make speech sound robotic or ambiguous
Ssml Markup Failures (GitHub)	Markup tags spoken aloud, ignored, or partially processed due to malformed or unsupported SSML
Voice Consistency Issues (GitHub)	Voice characteristics shift unexpectedly across languages, error states, or load-balanced instances

Total: 8 patterns

Related Goals

• Speech Recognition (GitHub) — the input-side mirror of synthesis: ASR mishearing rather than TTS mispronouncing
• Audio Handling (GitHub) — signal-path and acoustic-environment problems that affect the delivered audio quality regardless of what TTS produces
• Conversation Flow (GitHub) — dialog-level style failures (verbosity, unnatural conversational style) that sit above individual utterance synthesis