Expressive real-time speech synthesis
Generating natural, emotionally expressive speech in any voice with conversational latency, including cloning a speaker's voice across languages.
State of the art
Production speech-to-speech models output translated audio in the speaker's own voice with ~2–3 s latency (Google Meet, 2025).
- Prosody and emotion control in long-form narration still sounds flat or inconsistent
- Voice cloning enables fraud, forcing watermarking and consent controls
Who’s working on it
- ElevenLabs ↗Startup
The most widely deployed expressive TTS and voice-cloning API as of 2025-2026, and the reference system competitors benchmark against
- Cartesia ↗Startup
Sonic-3 (October 2025) delivers ~190ms end-to-end latency across 42 languages with laughter and emotional nuance, funded by a $100M Series B
- Albert GuResearcherCarnegie Mellon University (co-founder, Cartesia)
Co-invented the Mamba/state-space architectures that underpin Cartesia's sub-100ms streaming synthesis
- Hume AI ↗Startup
Octave 2 (October 2025) is a speech-language model covering 11 languages under 200ms with voice conversion and direct phoneme editing
- Sesame AI ↗Startup
Trained a Conversational Speech Model aimed at audio-first wearables; a16z led its Series A in February 2025
- Google DeepMind ↗Company
Ships Google Meet speech translation (2025) that speaks in a voice resembling the original speaker in near real time, at consumer scale
What to invest in
| Kind | Investment | Scale |
|---|---|---|
| research | Controllable prosody and character acting for long-form interactive narration | $$ $10M – $100M |
Depends on
A root node: no prerequisites modeled.