Anthropic and Theorem Use AI to Automate Formal Software Verification

Anthropic and its partner Theorem have demonstrated that large language models can automatically generate formal mathematical proofs to verify the correctness of existing software. This development marks a significant shift in how the industry approaches code safety, moving beyond manual review toward automated assurance.

The project utilized the Claude model to process test programs from 123 open-source libraries within Google's OSS-Fuzz security project. In a span of just a few weeks, the system produced approximately 180 million lines of proof code. This volume of work previously required substantial human resources and time, highlighting the potential for significant efficiency gains in the software development lifecycle.

Formal verification has historically been a niche practice reserved for high-stakes environments like avionics due to its prohibitive cost. For context, a team of specialists required 20 person-years to verify the small seL4 operating system kernel. By automating this process, Anthropic and Theorem aim to make rigorous verification a routine part of the standard development pipeline rather than an exceptional luxury.

The methodology involves translating crash tests into mathematical problems and analyzing compiled machine code. Claude generates the proofs, which are then checked by an independent verification program called Rocq. The team describes the process as fully autonomous, requiring minimal human intervention, although the underlying formal foundations were established by human engineers at Theorem.

A key financial and operational benefit emerged during the verification phase, where the model identified hundreds of bugs that conventional testing methods had missed. These errors often remained hidden because they only manifested under specific conditions, such as unusually large input files or specific access sequences. In one instance, the team discovered as many security bugs in a popular PDF reader in one week as are typically reported in an entire year.

However, the approach has strict technical limitations that affect its general applicability. The proofs are valid only under specific processor architectures, single-threaded execution, and inputs under 64 MB. These constraints mean the current proofs do not guarantee safety across all real-world hardware configurations and deployment scenarios.

The computational cost remains a major factor, with individual proofs consuming hundreds of millions of tokens. While total project costs were not disclosed, the team noted that newer models are becoming more efficient. Specifically, the Opus 5.5 model solved half the proofs using approximately 40 percent fewer tokens than the previous Opus 5 version, suggesting a trend toward lower marginal costs.

The initiative suggests that the market focus is shifting from whether AI can write code to whether it can prove that code is safe. If AI can guarantee correctness under defined conditions, developers may reduce time spent on manual reviews, lowering overall development costs. Theorem's roadmap targets critical infrastructure libraries such as glibc, OpenSSL, and cURL, indicating a plan to scale this technology to foundational software components.

The next phase of this technology will likely focus on expanding the scope of verified libraries and improving the efficiency of proof generation. As the tools mature, the industry may see a broader adoption of AI-driven verification, transforming it from a specialized task into a standard cost-saving measure for software providers.