Formal verification of neural network behavior
Mathematical proof that a neural network satisfies a specified property (never outputs X under condition Y) for all inputs in a defined set, rather than only on tested examples.
State of the art
SDP-CROWN (ICML 2025) verifies models up to 2.47M parameters; VNN-COMP 2025 benchmarks remain small vision, control and NLP models.
- Verification cost scales exponentially with network size for complete methods
- Behavioral rules like "do not harm a human" have no formal specification over raw inputs
- Properties proven on bounded input sets do not cover open-world inputs
Who’s working on it
- Huan ZhangResearcherUniversity of Illinois Urbana-Champaign
Leads alpha-beta-CROWN, which won VNN-COMP every year from 2021 through 2025 and defines the state of the art in bound-propagation verification
- alpha-beta-CROWN (Verified Intelligence) ↗Academic lab
Open-source verifier that topped all scored benchmarks at VNN-COMP 2025; proves robustness properties for networks up to a few million parameters
- International Verification of Neural Networks Competition (VNN-COMP)Program
Its sixth edition (2025) ran 8 teams over 16 regular and 9 extended benchmarks, and is the field's only standardised measure of what is provable at what scale
- Clark BarrettResearcherStanford University
Co-created Marabou, the SMT-based verifier used where bound propagation is too loose, and co-organises the benchmark standardisation effort
- Zico KolterResearcherCarnegie Mellon University
Originated the convex-relaxation approach to certified robustness that every current verifier descends from, and now also chairs OpenAI's safety committee
- DARPA Assured Neuro Symbolic Learning and Reasoning (ANSR)Program
Funds hybrid neuro-symbolic architectures designed so that safety properties are statable and provable, attacking the specification gap rather than the solver
- Daedalean ↗Startup
Swiss company that worked with EASA on Concepts of Design Assurance for Neural Networks (2020-2021), the first regulatory framework for certifying a neural network in a safety-critical aircraft system
What to invest in
| Kind | Investment | Scale |
|---|---|---|
| research | Scalable verifiers for transformer-scale models and compositional proofs over modules | $$ $10M – $100M |
| research | Formal specification languages for behavioral constraints of embodied agents | $$ $10M – $100M |