Continual learning without catastrophic forgetting
Learning algorithms that let a deployed model keep acquiring knowledge and skills from experience over months to years without erasing earlier competence or drifting in behavior.
State of the art
Nested Learning / Hope (NeurIPS 2025) improves continual-learning and long-context benchmarks; no frontier model updates its weights continuously in deployment (2026).
- Weight updates on new data degrade prior tasks and safety alignment (spurious forgetting)
- No method to validate a model whose parameters keep changing after release
- Per-user weight updates multiply serving cost and memory
Who’s working on it
- Ali Behrouz ↗ResearcherGoogle Research (and Cornell University)
First author of Titans (2025) and Nested Learning / Hope (NeurIPS 2025), the self-modifying architecture with a continuum memory system that reduced forgetting on continual-learning benchmarks
- Google ResearchCompany
Published the Nested Learning paradigm (2025) and the follow-on "sleep" consolidation work (2026) that distils short-term into long-term memory
- Bing LiuResearcherUniversity of Illinois Chicago
Long-running lifelong-learning programme; his in-context continual learner with an external learner (InCA, 2025) is the standard non-parametric baseline
- TrajectoryStartup
Raised a $15M seed in 2026 and then a Sequoia-led round to build a platform that keeps agentic models training on signals from live products
- Core AutomationStartup
Founded 2026 specifically for continual-learning models; filed to raise up to $500M, the largest single bet on the problem so far
- AnthropicCompany
Ships frozen frontier models with retrieval and long-context memory rather than online weight updates, which is the state of practice this capability has to displace
What to invest in
| Kind | Investment | Scale |
|---|---|---|
| research | Architectures with multi-timescale memory tested on year-long streaming benchmarks at frontier scale | $$$ $100M – $1B |
| research | Regression-testing and rollback methods for continuously updated models | $$ $10M – $100M |