Most people pick one model and ask it to do everything. We stopped doing that months ago, mostly because it kept running out. Our subscriptions hit their limits, and the one model doing everything also checked its own work, which is the one job a model is worst at.
So we split it. Claude plans and checks. Grok attacks whatever Claude just wrote. Codex builds the big pieces. And as of this week, Tīrtha, our own router, takes the routine building. We kept a ledger of who caught what. Here is what it shows, and what it doesn't show yet.
| Reviewer | Findings raised | Held up on re-check | Not in Claude's draft | Critical and new |
|---|---|---|---|---|
| Grok | 519 | 309 | 207 | 59 |
| Codex | 50 | 39 | 33 | 5 |
Logged Sep 25 to Oct 17. Cost per Grok review: median $0.021 over the last 30 ($2.32 total).
Grok: the cheap second opinion
Grok raised 519 findings between Sep 25 and Oct 17. When we re-checked them by hand, 309 held up. 207 of those were things Claude's own draft did not have, and 59 of the 207 were critical.
It makes mistakes too. 28 of its findings were simply wrong, and another 30 were wrong because our brief was missing something. At two cents a review, a reviewer that is sometimes wrong is still well worth having.
The last 30 Grok reviews cost $2.32 in total, a median of about two cents each. It is cheap because it never gets the repo. It gets a short brief with the claim and the evidence, around 2,300 tokens, and is told to break it. (We first wrote about this in September, in What Grok catches, what Codex catches, and what the pair costs us. That was two reviews. This is 746 logged findings.)
One example from today. Claude built a gateway fix and it passed 55 tests. A Grok attack that cost 17 cents found three cases where the fix would have blanked a correct answer. All three were real, and all three are fixed now.
Codex: the one that reads everything
Codex raised far fewer findings, 50, but 39 held up and 33 were new to Claude. The best one: it noticed that our live gateway was writing no usage rows at all for some requests, so billing was blind. Nobody else would have seen that without reading the code.
The depth costs a lot. A median Codex run uses about 84,000 tokens, and between Sep 22 and Oct 14 we hit Codex's usage limit 7 times, twice for days. Every routine job we hand Codex is a job that eats the window we need for the hard ones.
Tīrtha: the routine builder (just started)
The bet is simple: let Codex do what only Codex can do, and send the plumbing to a cheap, checked builder, so the subscriptions last. We ran a version of this in the summer, when a free cheap tier did 36 of 52 fleet tasks and we saw no Codex limit errors from June to August. The timing fits, but plenty else changed in those months, so we can't call it proof.
What we can't show yet
- Whether the whole team beats Claude working alone. We built that test and haven't run it.
- Whether moving the building to Tīrtha makes the subscriptions last longer. We will measure it now that it has started.
- The ledger itself has a weakness: most of its 746 rows were logged after the fact from saved reviews, and Claude judged which findings were already in its own draft. A model grading its own blind spots is exactly the kind of thing Grok would flag.
So, how do you split yours?
Do you use one model for everything, or give each one a job? Has anyone measured whether a cheap reviewer actually catches more than it costs you in false alarms? We'd like to compare notes, and we'll share our ledger format if it helps.
