OpenAI's 722 Manuscripts: A Forensic Audit of the Lean Verification Gap

"Glitch detected. Source traced."

Three hundred seventy-two problems. Seven hundred twenty-two manuscripts. Zero disclosed pass rate.

That is the entire technical payload of the story now moving through Crypto Briefing: that mathematicians are combing through OpenAI's AI-generated results on more than 300 problems, with a formal proof assistant — almost certainly Lean — sitting somewhere inside the loop. No benchmark table. No success percentage. No difficulty tier for the problems. No split between "end-to-end compiled" and "partially formalized." Just the verb combing, which is the most honest word in the whole report, because combing is what you do when you do not yet know what is real.

I have spent twenty-seven years watching systems declare victory before the audit closes. Based on my audit experience, a missing number is never an accident. When a lab has a figure it likes, the figure leads the headline. When it does not, the model becomes "unreleased" and the outputs become "manuscripts." That is not cynicism. That is reading the metadata before reading the marketing.

So let me do what the press release will not: reconstruct the verification architecture from the vocabulary, and mark every place where the claim outruns the evidence.

To understand why this matters, you have to understand the benchmark OpenAI is implicitly being measured against, and it is not a chat benchmark.

In July 2024, Google DeepMind's AlphaProof and AlphaGeometry systems reached silver-medal level at the International Mathematical Olympiad, solving four of six problems, with AlphaProof producing proofs in Lean — a formal proof assistant whose Mathlib library now holds on the order of a million lines of machine-checked mathematics. That result reset the standard. Before AlphaProof, "AI does math" meant fluent text that sounded like a proof. After AlphaProof, it meant a Lean kernel that either accepts the term or rejects it. There is no middle ground in a kernel. This is the line OpenAI is now walking toward.

The mechanism behind both efforts is the same family: a large language model generates candidate reasoning, and a verifier — Lean — supplies a binary reward. Compile passes, reward is one. Compile fails, reward is zero. This is Reinforcement Learning with Verifiable Rewards, RLVR, and it is the single most important training development of the last two years, because it sidesteps human preference labeling, which is expensive, slow, and gameable. Mathematics is the ideal RLVR domain precisely because ground truth is cheap to check and impossible to flatter.

The Mathlib library is the detail that gives this story its spine. Formalizing even elementary mathematics in Lean is laborious — a single undergraduate theorem can take an expert hours to encode correctly — which is why the existence of a large, machine-checked corpus is a precondition for any AI that wants to learn from verification rather than from text. RLVR does not work without a verifier, and the verifier does not work without a library. The whole edifice rests on a decade of human formalization labor that no lab has replaced. When OpenAI claims progress in formal math, it is standing on that unpaid foundation whether it says so or not.

OpenAI's o-series already established that long chain-of-thought reasoning scales. What it never established, publicly, was formal-math competence at DeepMind's level. This event — an unreleased model, hundreds of manuscripts, independent mathematicians reviewing — is the first attempt to claim that territory. It arrives in a bull market, which matters: in euphoria, the market reads "manuscripts" as "theorems" and prices the narrative before the kernel ever runs. My job is to price the kernel.

A note on provenance, because sourcing is evidence too. Crypto Briefing is almost certainly relaying a first-hand account from a science desk or a lab channel, which means we are reading a copy of a copy. Every layer of relay strips a technical detail and adds a narrative flourish. By the time "AI-generated results" reaches a crypto audience, the caveats that would have made it accurate have been optimized away for clarity. That is not a conspiracy. It is just how compression works.

The first thing a forensic reader does is divide.

Seven hundred twenty-two manuscripts over three hundred seventy-two problems is roughly 1.94 attempts per problem. That ratio is the whole confession. A system that could reliably prove a theorem once would not need two passes per problem. Two passes per problem is the signature of best-of-n sampling — generate multiple candidates, run each through the verifier, keep whichever compiles. That is a filtering architecture, not a solving architecture. It tells you the model is a high-variance generator and Lean is the low-variance filter doing the actual epistemic work. The intelligence is not in the model. The intelligence is in the kernel, and the kernel was built by humans.

Now the wording trap. If the outputs were complete Lean proofs, the report would call them proofs. It calls them manuscripts. The word "manuscript" is a natural-language artifact, not a formal one. A manuscript can contain a beautiful argument that has never touched a compiler. This raises the probability that a substantial fraction of the 722 outputs are prose sketches — proof narratives — with only fragments formalized, or formalized by human hands after the fact. The 1.94 ratio supports this reading: if every output were a self-contained Lean file, the reporting would quote a compile success rate, because that number would be trivially available. Its absence suggests the number is not flattering, or does not exist in a single clean form.

Reverse-inferring the success rate from silence is legitimate methodology. When a lab with world-class communications discipline omits a metric, the omitted metric is the one that would embarrass. If OpenAI had passed, say, sixty percent of 372 problems end-to-end in Lean, that figure would be the headline. The realistic band, given the "combing" language and the unreleased status, is low. I will not invent a number. I will state the direction: below the level that would justify a launch, which is exactly why there is no launch.

Now the question the report refuses to ask: how hard were the problems?

Difficulty is the entire scoreboard in mathematics, and it is completely absent here. Three hundred seventy-two problems could mean olympiad-level, where a silver-medal system already exists; or undergraduate-textbook-level, where the claim is unremarkable; or frontier open problems, where a single verified result would be a genuine scientific event. Without the difficulty tier, the number 372 is decoration. A system that proves 372 textbook lemmas has demonstrated a search engine. A system that proves one open conjecture has demonstrated something else entirely. The report gives us the count and withholds the calibration, which is like publishing a test score without the test.

The second unasked question is autonomy. There are two very different architectures hiding under "AI-generated." In the first, the model writes Lean code, compiles it, reads the errors, and iterates — a genuine agentic loop where the model is the author and the kernel is the critic. In the second, the model produces a natural-language idea and a human formalizes it, meaning the human did the hard, verifiable part and the model did the easy, fluent part. The difference between these two architectures is the difference between a breakthrough and a parlor trick, and the report does not distinguish them. A reader who assumes the first when reality is the second has been misled without anyone lying.

Here is the part the crypto press is missing entirely, and it is why I am writing this for a crypto audience at all.

The verifier stack being demonstrated here is the same stack that underwrites smart-contract security. Lean and its cousin Coq are not academic toys. They are the formal backbone of an industry that has spent a decade trying to make software correctness provable rather than probable. If AI can generate machine-checked proofs, the cost curve of formal verification collapses — and formal verification is the single largest unsolved cost center in smart-contract auditing.

I know this curve from the inside. In 2017, before the Ethereum mainnet launch, I spent forty-eight hours straight inside a pre-sale script and found an integer overflow in Solidity that would have drained roughly 0.05 percent of early funds. I caught it by reading bytecode, not by reading the whitepaper. That experience taught me the founding law of my career: press releases describe intent, code describes reality, and the two almost never agree. The tools I used then were manual — eyes, disassemblers, patience. What OpenAI is gesturing at is the automation of exactly that patience.

Three years later, during DeFi Summer 2020, I identified a flash-loan attack vector in Compound's interest-rate model three hours before major exchanges halted trading. I wrote a 3,000-word forensic report on the reentrancy exposure in the cToken logic while the rest of the timeline posted panic. That report drew fifty thousand views in twenty-four hours because it explained the mechanism instead of amplifying the fear. Forensics scale; panic does not. An AI that can produce formal proofs of contract invariants would industrialize the mechanism-level explanation that once took me a sleepless night to write by hand. That is the real headline hiding inside a math story.

But — and this is where I break from the bull-market reading — the leap from "proving olympiad-style lemmas" to "proving production contract invariants" is enormous, and the report does not even acknowledge it exists. Olympiad problems are curated to be solvable. Production contracts are adversarial by design, written by humans who did not want to be verified, and their specification — what the contract is supposed to do — is often the hardest part, not the proof. A prover cannot verify what cannot be specified, and specification is a human, political, and economic act. Lean checks proofs against statements. It does not tell you the statement was the right one. Every catastrophic DeFi exploit in history — including the ones I have dissected — was a gap between what the code did and what people assumed it did, not a gap in proof technique.

Let me trace the cost structure, because cost is where narratives die.

The o-series is defined by long chain-of-thought. A single hard reasoning query can emit thousands to tens of thousands of tokens before it stops. Multiply that by 722 manuscripts, each possibly the survivor of multiple sampled attempts, and you have an inference bill that dwarfs ordinary chat. Add the verifier: Lean proof-checking is CPU-intensive, not GPU-parallel, so every candidate proof consumes scarce general-purpose compute before it can be scored. Liquidity draining. Logic broken. — that is what a failing rollout looks like at scale, a hundred candidates generated and discarded for every one that compiles. A lab does not burn this much compute on an unreleased model unless it is buying something it cannot buy cheaper: the credibility of a new capability tier.

And that is the strategic read. This is not a product. This is a pricing argument. OpenAI is trying to anchor "verifiable reasoning" as a scarce, premium commodity so that the next generation of models — and their per-token prices — inherit a legitimacy that benchmarks alone could not confer. In a bull market, the market will happily pay for that story. Read it as a bond prospectus, not a discovery announcement.

Watch the tokens that pump on this headline, too. Exchange volume anomaly flagged — whenever a lab announces an unreleased capability, a predictable cohort of thinly-related assets catches a bid on narrative alone, and the volume pattern is identical every cycle. The pumps are not the news. The pumps are the reaction to the news, and they decay faster than the news does.

Now the competitive frame, because capability claims are meaningless without a baseline.

DeepMind is the incumbent here and it earned the position. AlphaProof's IMO 2024 result is public, reproducible, and tied to a kernel. OpenAI's claim is private, unreviewed in the sense that matters, and tied to manuscripts. On formal mathematics specifically, OpenAI is the challenger, not the leader, and the report contains no evidence of a reversal. What OpenAI does hold is the largest developer surface and the most aggressive reasoning roadmap. If it converts that into a Lean-focused RLVR program — and hires the formal-mathematics talent it currently lacks — a catch-up inside twelve to eighteen months is plausible. Plausible is not proven. The honest scorecard reads: DeepMind ahead on formal proof, OpenAI ahead on general reasoning and distribution, and nobody has published a head-to-head on the same problem set.

There is a detail buried in the sourcing that the crypto-native reader should not skip. This story surfaced through Crypto Briefing. A general-purpose lab's formal-mathematics progress would normally route through a science desk. The fact that it landed in a crypto outlet is not noise. The crypto industry is one of the few sectors that pays real money for verifiable correctness today — smart-contract auditing, bridge invariants, consensus proofs — and the market is beginning to price AI-assisted formal verification as an infrastructure primitive rather than a research curiosity. Watch the tools before you watch the tokens.

There is a second-order risk that no one in the coverage has named, and it comes straight out of the metadata layer.

In 2021 I spent two weeks reverse-engineering the off-chain metadata retrieval path of the Bored Ape Yacht Club contract. On-chain, the token looked immutable. Off-chain, the traits lived on a server the team controlled. The scarcity was a promise, not a property. NFT metadata mismatch found — and it is the same structural failure mode now hiding in AI math claims: the surface says "proven," the substrate says "curated." A Lean-checked lemma is a property. A manuscript selected from two samples and narrated for a reporter is a promise. The report gives us promises and lets the reader assume properties.

Then there is the labor question, and it is not sentimental. Seven hundred twenty-two manuscripts require humans to comb through them. Those humans are mathematicians — the scarcest, most expensive reviewers on earth. OpenAI has effectively outsourced its evaluation layer to expert labor and framed it as collaboration. I have seen this pattern before: the free-audit pipeline. Projects ship code, crowdsource the bug-finding, and keep the upside. If AI begins generating proof candidates faster than humans can adjudicate them, the bottleneck of mathematics shifts from discovery to review, and the profession's workload inverts. The Terra collapse taught me the same lesson in a different register: the system's fragility was never in the visible peg, it was in the incentive layer nobody was paid to audit until it was too late.

That brings the security frame into focus, and it is the one that should make a crypto reader pay attention.

Formal mathematical reasoning is a proxy for general verifiable reasoning, and general verifiable reasoning is dual-use. The same capability that proves a lemma can, in principle, model a molecular pathway or a network attack surface. OpenAI's own safety frameworks acknowledge that strong reasoning is the shared upstream of biological and cyber risk. A math result is benign; the reasoning substrate that produces it is not automatically benign. The report does not touch this. It does not need to — but a market pricing "AI solves math" should at least price the fact that the same engine reads both ways.

Let me be precise about what I am not saying. I am not saying the results are fake. Lean, if it is genuinely in the loop, is incorruptible — the kernel does not care about narrative. I am saying the reporting has given us the parts of the system that cannot be checked and withheld the part that can. That is a metadata mismatch at the story level, and the fix is the same as it is in contracts: publish the verification pass rate, publish the problem set, publish the fraction that compiled end to end. Anything less is a press release wearing a lab coat.

One more layer, and then I will stop taking the system apart.

The infrastructure under all of this is fragile in ways the narrative ignores. RLVR requires massive rollout sampling — generate, verify, discard, repeat — which means training compute scales with the rejection rate, not the success rate. A model that fails ninety percent of attempts still pays for a hundred percent of the attempts. The worse the pass rate, the higher the cost — failure is not free in verifiable reinforcement learning; it is the most expensive part of the bill. That is a strange economic property, and it is why I keep insisting the undisclosed pass rate is not a footnote. It is the balance sheet.

And here is where my oracle skepticism becomes relevant. DeFi's oldest wound is not bad code — it is stale data. Oracle feed latency is the Achilles' heel of every on-chain system that reads the world, and no amount of proof generation fixes a feed that is late. You can formally verify that a liquidation function does exactly what its specification says and still get liquidated wrongly, because the specification trusted a price that had not updated. Formal verification makes code provably correct against its assumptions. It says nothing about whether the assumptions were true. Every audit I have ever run has taught the same thing: the bug is almost never in the logic. It is in the boundary where the logic meets reality. AI proof generation lives entirely on the logic side of that boundary. That is why it is useful, and that is why it is not enough.

Here is the angle nobody is publishing, and it is uncomfortable for everyone.

The story is being read as "OpenAI does math." The story is actually "OpenAI bought an evaluation." The unreported fact is the direction of the transaction: mathematicians are not being tested by the model; the model is being tested by mathematicians, who are doing that work for reputational and collaborative reasons rather than commercial ones. That inverts the usual narrative of AI displacing labor — here, human expertise is the scarce input and the model is the applicant. Read it that way and the "722 manuscripts" become less a triumph than a stack of job applications awaiting review.

The second blind spot is survivorship. We will hear about the manuscripts that impress. We will not hear about the ones that failed, because failures are not manuscripts — they are deleted tokens. The disclosed set is a curated museum, not the full dig site. Any inference about capability drawn from a curated set is a metadata claim, not a measurement. And metadata, as I learned with the apes, is where systems lie without technically lying.

The third blind spot is the one the industry will least want to hear: if the pass rate is low, the correct description of this event is not "OpenAI advances mathematics." It is "OpenAI funds an expensive search that occasionally lands." That is still interesting. It is not what the headline sells.

The next signal to watch is not a demo. It is a number: the end-to-end Lean compile rate across the full 372-problem set, published by someone with no incentive to round up. If it appears, the field has a real baseline and the marketing collapses into measurement. If it never appears, ask why. A capability that cannot be audited is not a capability. It is a narrative — and narratives do not compile.