On Oct 6, OpenAI put 722 mathematics manuscripts online, and only one had a human edit before release. Terence Tao calls this proof indigestion: more AI-written mathematics than the people who must judge it can absorb. A machine can now write a proof-shaped manuscript faster than anyone can check it. So the sharp question is who checks the checker.

TL;DR

  • Checks come in levels: a machine check, a named human review, released artifacts. OpenAI's three Oct 26 headline results got different mixes, from Sogge (machine only) to Yang-Mills (artifacts withheld pending review).
  • The catalog shows no check status per paper, and The Atlantic says some appear not to prove their claims.
  • Lean checks the proof steps, not whether the claim says what the mathematician meant. That second check is human.

OpenAI's 41-page catalog of 722 papers marks no check status for any of them

OpenAI published 722 manuscripts covering 372 result families in a public GitHub repo, written by an unreleased internal model. The claims range from Khot's unique games conjecture to the quasi-Riemann hypothesis, a weaker version of the Riemann hypothesis. The Atlantic reported that some of the papers appear not to prove what they claim.

Even a single headline can hide two check states. The Sept 30 version of quasi-Riemann, with zeros between 1/8 and 7/8, is Lean-verified. The later version, with a wider strip from 1/12 to 11/12, is not formally verified. A result's name does not tell you which version was checked.

Three things to look for before you believe an AI-proved headline

OpenAI's three Oct 26 headline results:

Result What it claims Who has checked it
Sogge local smoothing A proof of Sogge's local smoothing conjecture, about how waves focus and fade, in every dimension of at least 2 A machine only: Lean-formalized Oct 19. No named human re-check
ETH disproof A disproof of the Exponential Time Hypothesis for 3-SAT A machine and named experts: Lean-verified Oct 17, shepherded by the hypothesis's original proposers Impagliazzo and Paturi plus Ryan Williams. OpenAI also committed to funding an open community conference on the claim
Yang-Mills and QCD existence Existence proofs for Yang-Mills theory and QCD, two quantum physics theories No outside review yet: Lean formalization finished Oct 16, artifacts withheld pending review by a group led by Nobel laureate David Gross

Try this: open the primary announcement of any AI-proved result and look for:

  • a Lean status for the result
  • a named group that has finished reviewing it
  • released files you can inspect

Search the announcement text for the word Lean and for a reviewer's name, and check whether it links any files. Each missing item lowers how far you should trust the result. Only when all three are present, treat it as publicly checked.

Run the list on the Yang-Mills row. A Lean status: yes, the formalization is finished. A named group that has finished reviewing: no, the Gross-led group's review is pending. Released files: no, they are withheld. One of three items is met, so treat it as unchecked in public.

Only ETH has named people already working on the claim.

What a Lean check covers, and what it leaves to a human

Lean is a proof assistant. It checks that every step follows from the claim as stated, and it cannot judge whether the stated claim says what the mathematician meant.

Lean is the checker, and the human who reads the formal statement and confirms it matches the intended conjecture is the checker's checker. Picture a mathematician reading the Lean version of the ETH disproof and confirming it formalizes the real Exponential Time Hypothesis for 3-SAT, not a nearby, easier statement.

If you write software, you know this shape. This analogy is ours, not OpenAI's or Tao's: a green test suite shows the code matches the tests, and only a person can say whether the tests describe what the customer wanted. Lean plays the test runner, and the formal statement plays the test.

Tao's complaint behind proof indigestion is that the machines skip the explaining and judging that make a result usable. There is money for evaluation: the Simons Foundation has put $15M into it, $5M of it from OpenAI.

Who to wait for before you share the next one

When the next AI-proved headline lands, run the three-item list and ask which version of the claim it describes. Then wait for the humans. For Yang-Mills that means the report from the Gross-led group. For ETH it means the open community conference OpenAI committed to funding.

Dan writes the AI explainers on super-intelligence.rocks, where this piece first appeared.