SciFi Futures
← Capabilities
Computing & Information/Component technology

Photorealistic real-time avatars

Lifelike 3D representations of a person that track their face, eyes and body in real time, indistinguishable from video, captured with consumer hardware.

State of the art

Codec Avatars with Gaussian splatting rendered photoreal faces in the 2023–2024 Zuckerberg–Fridman demo on a multi-GPU workstation.

What’s blocking it
  • Rendering cost far exceeds standalone headset GPUs
  • High-quality capture still needs multi-camera rigs rather than a phone scan
  • Headset-mounted sensors see little of the face to drive expressions

Who’s working on it

  • Yaser SheikhResearcher
    Meta Reality Labs Research, Pittsburgh

    Directs Codec Avatars; said in 2024 the project had gone from ten miracles away to five, with the 2.0 demo needing four RTX 4090s

  • Meta Reality LabsCompany

    Published Universal Relightable Gaussian Codec Avatars, phone-depth-scan avatar generation, HairCUP (2025) and SqueezeMe (SIGGRAPH 2025) distilling Gaussian avatars onto phones

  • Bernhard KerblResearcher
    Inria GRAPHDECO (now TU Wien)

    First author of 3D Gaussian Splatting (SIGGRAPH 2023), the representation that every real-time photoreal avatar system switched to

  • Apple ↗Company

    Ships Personas in visionOS as the only consumer-shipping real-time volumetric self-representation, at visibly lower fidelity than research results

  • The largest commercial photoreal avatar business, but asynchronous video rather than real-time tracked telepresence, which marks where the market actually is

  • Soul MachinesStartup

    New Zealand company shipping real-time animated digital humans for customer interaction, driven by a behavioural model rather than a captured performer

What to invest in

KindInvestmentScale
researchPhone-capture avatar models and efficient neural rendering on mobile GPUs$$ $10M – $100M

Depends on

On-device neural inference accelerators
Real-time neural rendering on wearable hardware
TRL 8

Required by

No other capability builds on this directly.

Sci-fi concepts in reach

The MetaversegatingdirectSelf-aware supercomputer (Mike)direct

Sources