SciFi Futures
← Capabilities
Computing & Information/Foundational science

Continual learning without catastrophic forgetting

Learning algorithms that let a deployed model keep acquiring knowledge and skills from experience over months to years without erasing earlier competence or drifting in behavior.

State of the art

Nested Learning / Hope (NeurIPS 2025) improves continual-learning and long-context benchmarks; no frontier model updates its weights continuously in deployment (2026).

What’s blocking it
  • Weight updates on new data degrade prior tasks and safety alignment (spurious forgetting)
  • No method to validate a model whose parameters keep changing after release
  • Per-user weight updates multiply serving cost and memory

Who’s working on it

  • Ali Behrouz ↗Researcher
    Google Research (and Cornell University)

    First author of Titans (2025) and Nested Learning / Hope (NeurIPS 2025), the self-modifying architecture with a continuum memory system that reduced forgetting on continual-learning benchmarks

  • Google ResearchCompany

    Published the Nested Learning paradigm (2025) and the follow-on "sleep" consolidation work (2026) that distils short-term into long-term memory

  • Bing LiuResearcher
    University of Illinois Chicago

    Long-running lifelong-learning programme; his in-context continual learner with an external learner (InCA, 2025) is the standard non-parametric baseline

  • TrajectoryStartup

    Raised a $15M seed in 2026 and then a Sequoia-led round to build a platform that keeps agentic models training on signals from live products

  • Core AutomationStartup

    Founded 2026 specifically for continual-learning models; filed to raise up to $500M, the largest single bet on the problem so far

  • AnthropicCompany

    Ships frozen frontier models with retrieval and long-context memory rather than online weight updates, which is the state of practice this capability has to displace

What to invest in

KindInvestmentScale
researchArchitectures with multi-timescale memory tested on year-long streaming benchmarks at frontier scale$$$ $100M – $1B
researchRegression-testing and rollback methods for continuously updated models$$ $10M – $100M

Sci-fi concepts in reach

Self-driving carsindirectArtificial Friend companion robotgatingdirectHAL 9000 ship computergatingindirectPositronic robots bound by the Three LawsgatingindirectThe Young Lady's Illustrated PrimergatingdirectAI cybrid (AI persona in a grown human body)indirectCulture Minds (benevolent superintelligences)indirectSelf-organizing nanobot swarmdirectTechnoCore AI civilizationindirectSelf-aware supercomputer (Mike)directFarcaster portal networkindirect

Sources