Photorealistic real-time avatars
Lifelike 3D representations of a person that track their face, eyes and body in real time, indistinguishable from video, captured with consumer hardware.
State of the art
Codec Avatars with Gaussian splatting rendered photoreal faces in the 2023–2024 Zuckerberg–Fridman demo on a multi-GPU workstation.
- Rendering cost far exceeds standalone headset GPUs
- High-quality capture still needs multi-camera rigs rather than a phone scan
- Headset-mounted sensors see little of the face to drive expressions
Who’s working on it
- Yaser SheikhResearcherMeta Reality Labs Research, Pittsburgh
Directs Codec Avatars; said in 2024 the project had gone from ten miracles away to five, with the 2.0 demo needing four RTX 4090s
- Meta Reality LabsCompany
Published Universal Relightable Gaussian Codec Avatars, phone-depth-scan avatar generation, HairCUP (2025) and SqueezeMe (SIGGRAPH 2025) distilling Gaussian avatars onto phones
- Bernhard KerblResearcherInria GRAPHDECO (now TU Wien)
First author of 3D Gaussian Splatting (SIGGRAPH 2023), the representation that every real-time photoreal avatar system switched to
- Apple ↗Company
Ships Personas in visionOS as the only consumer-shipping real-time volumetric self-representation, at visibly lower fidelity than research results
- Synthesia ↗Startup
The largest commercial photoreal avatar business, but asynchronous video rather than real-time tracked telepresence, which marks where the market actually is
- Soul MachinesStartup
New Zealand company shipping real-time animated digital humans for customer interaction, driven by a behavioural model rather than a captured performer
What to invest in
| Kind | Investment | Scale |
|---|---|---|
| research | Phone-capture avatar models and efficient neural rendering on mobile GPUs | $$ $10M – $100M |
Depends on
Required by
No other capability builds on this directly.