Voice Synthesis

Text-to-speech output sounds wrong when the synthesis layer applies a default, one-size-fits-all rendering to text that actually needs domain-specific pronunciation, content-appropriate emotion, or a stable persona, and none of those needs are encoded as explicit metadata the TTS engine can act on. Brand names, acronyms, and heteronyms fall outside a general grapheme-to-phoneme model’s training distribution; emotional register and disfluency need to match message content and persona rather than a single fixed voice style; and markup, waveform generation, and voice selection all have independent failure modes that surface as audible artifacts, mispronunciation, or an inconsistent-sounding agent.

Key Takeaways

• 8 patterns split into three clusters: pronunciation accuracy (brand terms, general pronunciation, prosody), synthesis mechanics (audio artifacts, SSML markup), and persona/affect calibration (emotional tone, disfluency, voice consistency).
• Custom pronunciation lexicons cut name/brand mispronunciation errors by roughly 80%, and domain-term accuracy without hints runs only 50-70% — the single highest-leverage fix in voice synthesis is lexicon coverage, not a better base TTS model.
• SSML tag processing succeeds only 70-85% of the time across engines, and 5-10% of failures result in raw markup being spoken aloud verbatim — a jarring, immediately noticeable defect distinct from subtler mispronunciation.
• Emotional appropriateness in production TTS runs only 50-70%, and inappropriate tone is linked to a 25% reduction in user trust, showing that affect mismatch is as consequential as outright mispronunciation.

Scope

• Pronunciation Accuracy — brand-term-mispronunciation, pronunciation-errors, prosody-mismatch. Grouped because all three concern the TTS engine’s default grapheme-to-phoneme and rhythm rules producing an incorrect or unnatural rendering of otherwise-correct text — wrong phonemes for brand/domain terms and names, or wrong stress/intonation/pacing for sentence-level meaning.
• Synthesis Mechanics — audio-artifact-generation, ssml-markup-failures. Grouped because both are pipeline/engine-level defects (waveform discontinuities, markup parsing failures) rather than linguistic mispronunciation — the text-to-phoneme mapping can be correct while the audio production or markup interpretation still fails.
• Persona and Affect Calibration — emotional-tone-mismatch, disfluency-calibration-failures, voice-consistency-issues. Grouped because all three concern whether the voice’s emotional register, natural hesitation, and identity stay matched to persona and context across a conversation, rather than any single utterance’s phonetic correctness.

When Voice Synthesis Matters

• The agent must speak brand names, acronyms, technical jargon, or names not in a general TTS lexicon, where mispronunciation directly damages brand perception or trust
• Message content spans emotionally distinct categories — fraud alerts, approvals, denials, appointment reminders — where a mismatched tone (cheerful fraud alert, flat congratulations) is itself a user-facing failure, not just an accuracy one
• The deployment uses SSML markup, multiple languages, or load-balanced TTS infrastructure, where markup-processing failures or voice-selection drift can silently break the intended output

Cross-Pattern Insight

The dominant mitigation across all 8 patterns is making an implicit rendering decision explicit and auditable: custom lexicons and SSML phoneme tags make pronunciation an authored decision instead of a model guess; emotion tags attached to message templates at authoring time make tone a deliberate content decision instead of a runtime inference; and a single canonical voice-persona specification that every code path must reference eliminates the “different voice for errors” class of inconsistency. In every pattern, the failure mode is the TTS engine silently defaulting — to its trained pronunciation rules, its uniform prosody, or whatever voice a given code path happens to select — and the fix is removing that default by supplying explicit metadata upstream of synthesis.

Frequently Asked Questions

What’s the difference between brand-term mispronunciation and general pronunciation errors?

Brand-term-mispronunciation focuses specifically on brand names, acronyms, and version/number formats unique to a product or company. Pronunciation-errors covers the broader class — personal names, heteronyms (read/read), foreign words — that any TTS deployment encounters regardless of brand. Both are fixed the same way: custom lexicons and SSML phoneme overrides checked before default synthesis.

What causes SSML markup to sometimes get read aloud instead of applied?

SSML-markup-failures attributes literal-tag-spoken failures to malformed tags, unsupported engine features, or encoding/escaping issues in dynamically inserted content — the pattern reports 5-10% of SSML usage incidents result in literal tags being spoken, and recommends pre-send validation against an engine-specific capability matrix rather than assuming one SSML dialect works everywhere.

Can a single TTS voice handle both good news and bad news appropriately?

Not without explicit emotion tagging. Emotional-tone-mismatch documents cheerful delivery of fraud alerts and flat delivery of approvals as a common failure, with emotional appropriateness in production running only 50-70% — the fix is classifying each message’s required emotional register at template-authoring time, not relying on the TTS engine to infer it from content alone.

What can cause a voice agent’s voice to change mid-conversation?

Usually unintentional infrastructure behavior: different TTS instances behind a load balancer, a language switch without matched cross-language persona settings, or a different voice hardcoded for error messages versus success messages. Voice-consistency-issues reports 80%+ of users detect voice changes and 60% of those report confusion, recommending session-pinned TTS routing and a centralized voice-persona configuration all code paths must use.

Patterns

Total: 8 patterns

Related Goals

• Speech Recognition — the input-side mirror of synthesis: ASR mishearing rather than TTS mispronouncing
• Audio Handling — signal-path and acoustic-environment problems that affect the delivered audio quality regardless of what TTS produces
• Conversation Flow — dialog-level style failures (verbosity, unnatural conversational style) that sit above individual utterance synthesis