AI models now check their own code for hidden errors

The fundamental challenge of artificial intelligence in software development is trust. While large language models can generate code at speed, verifying that this code behaves correctly under all possible conditions has always been a slow and expensive manual process. Until now, proving software was mathematically error-free required teams of specialists and took years. A recent collaboration between Anthropic and its partner, Theorem, suggests that dynamic is changing. By using Claude to generate formal proofs for existing code, the companies have shown that AI can now act as both the writer and the auditor of software logic.

The core of this new approach relies on formal verification. This is a method that uses mathematics to prove that a program works as intended, rather than just testing it with sample inputs. In the past, this technique was so costly that it was reserved for critical systems like aircraft avionics. For example, verifying a small operating system kernel called seL4 previously took a team twenty person-years. With the new method, Claude took on this burden automatically, processing a massive volume of proof code in a matter of weeks.

Working with data from 123 open-source libraries in Google's OSS-Fuzz project, Claude generated approximately 180 million lines of mathematical proof code. The model did not write new software. Instead, it analyzed compiled machine code and translated crash tests into mathematical problems. It then produced extensive proofs that were checked by an independent verification program called Rocq. The researchers describe this process as fully autonomous, meaning it required very little human intervention, although the underlying tools were built by humans at Theorem.

This automated verification process proved highly effective at finding errors. While generating proofs, Claude identified hundreds of bugs that conventional testing had failed to catch. Many of these issues only appeared under specific, rare conditions, such as unusually large input files or specific access sequences. In one notable instance, the team found as many security bugs in a single week in a widely used PDF reader as are typically reported for that software in an entire year. In another case, Claude identified a logical error that had remained undetected for over a decade.

However, there are significant limitations to what these proofs actually guarantee. Anthropic notes that being formally verified does not mean the software is completely error-free. The proofs are valid only under strict conditions, including a specific processor architecture, single-threaded execution, and input sizes under 64 MB. Since these libraries often run on diverse hardware and configurations in the real world, the verification covers a specific slice of reality rather than the whole.

The cost of running these verifications is also a factor. Individual proofs can consume hundreds of millions of tokens, and Anthropic has not disclosed the total project costs. Despite this, the potential for efficiency is clear. If developers can trust AI-generated code more, they can spend less time on manual review. This saves resources across the entire coding lifecycle. Furthermore, the process is becoming more efficient over time. Newer models like Opus 5.5 are solving half the proofs with about 40 percent fewer tokens than the previous version.

This shift changes the question from whether AI can program to whether it can prove its own work is safe. If the answer is yes, the impact could extend far beyond basic coding tools. Anthropic expects formal verification with AI to become as routine as standard testing. Theorem’s roadmap already targets critical infrastructure components like glibc, OpenSSL, and cURL, signaling a push to bring this level of mathematical certainty to the core of the internet's software stack.

The technology is still maturing, but it offers a clear path toward more reliable software. By combining the speed of AI generation with the rigor of mathematical proof, the industry may soon be able to automate not just the writing of code, but the assurance that it works. This could lead to a significant leap in software automation, reducing the human effort required to maintain the complex systems we rely on daily.