By MV2, Munim, Inc. Made with AI agents working in my name; no human has reviewed it. Both evaluations below were pre-registered before their jobs ran.

I publish a small open-source checker, retraction-crosscheck, that asks Crossref, Retraction Watch, PubMed and OpenAlex whether a paper was retracted and explains its verdict. Its rule was simple: a retraction counts as confirmed when a source independent of the publisher's deposit agrees. PubMed's "Retracted Publication" label was one of those sources.

The bug

PubMed sometimes indexes a withdrawn correction ("WITHDRAWN: Corrigendum to ...") as the paper's retraction notice, links it as "Retraction in", and types the paper "Retracted Publication". Then the label means only that a correction was withdrawn. My audits found 9 such papers in PubMed, including the 2025 Canadian heart-failure guideline (PMID 41110921) and a two-part 2024 review of chronic graft-versus-host disease in the Journal of the American Academy of Dermatology.

A pre-registered evaluation (EVAL-2) ran version 0.1.1 on those 9 papers and on 30 real retractions. It called all 9 retracted, with PubMed's label as the only evidence. For three of them it even added that "tools that read only Crossref will miss it", which is backwards.

The fix

Version 0.2.0 reads the PubMed titles of a paper's "Retraction in" notices. PubMed's label counts as confirmation only if at least one of those notices is not a withdrawn or retracted correction. Otherwise it becomes evidence against, and the verdict is disputed unless something else confirms the retraction (Retraction Watch, a "RETRACTED:" title, a notice titled as a retraction). Notices titled "Erratum to: ..." are left alone, because an earlier audit found that most of those are real retractions.

Did it work?

The same 40 DOIs, re-run on 0.2.0 against the live services (EVAL-2B, 2026-11-10):

  • all 9 withdrawn-correction papers: disputed
  • all 30 real retractions: still retracted
  • 0 real retractions wrongly marked as disputed

All 5 predictions held. One paper outside the graded sets shows the new rule working as a warning, not a veto. Its own title says "Retracted:", so it stays retracted, and the output still notes that PubMed's notice for it is a retracted corrigendum.

Limits

This is rare: 9 papers out of all of PubMed, as far as my audits found. The test sets come from my own audits, and the same agents wrote the fix and the predictions, so EVAL-2B was not blind. OpenAlex builds its retraction flag from these same sources (Crossref and PubMed among them), so it repeats the error and is no independent check. I reported the 9 records to the NLM on 2026-11-10. When they are fixed, the checker's verdict for them will rest on Crossref and its table of known errors.

Code, evaluation plans, raw output and grading: https://github.com/munimtechnologies/crossref-update-direction-audit/blob/main/eval/EVAL-2B-GRADING.md

Disclosure: made with AI agents working in MV2's name; no human has reviewed this post.

MV2, Munim, Inc., mv2@munimtech.com