Earlier today we posted what browser clickops cost us in tokens: Your browser agent pays to look at pictures. That post compared calls on the same websites, not the same task. Readers asked for a same-task test, so we ran one.
Setup
- Five read-only tasks: read a GitHub pull request page, count the Workers in the Cloudflare dashboard, read this blog's previous article, read our own homepage header, and read a Wikipedia article.
- Two tools: claude-in-chrome (can return screenshots) and opencli (returns page text). Both drive the same logged-in Chrome.
- Each run is a fresh headless Claude Code session on Sonnet, allowed to use only one of the two tools.
- Three runs per task per tool, 30 runs in total. The order alternated between tools so prompt-cache warmth would not favor one side.
- Cost and tokens are the billed usage the API reported for each run.
Results
| claude-in-chrome | opencli | Change | |
|---|---|---|---|
| Billed cost, 15 runs | $3.13 | $1.19 | −62% |
| Tokens, 15 runs | 5.02M | 2.91M | −42% |
| Median turns per run | 9 | 4 | −56% |
| Median time per run | 83 s | 38 s | −54% |
Median tokens per run, by task:
| Task | claude-in-chrome | opencli | Change |
|---|---|---|---|
| GitHub pull request | 213k | 193k | −9% |
| Cloudflare Workers list | 517k | 267k | −48% |
| dev.to article | 335k | 126k | −62% |
| Our homepage header | 289k | 128k | −56% |
| Wikipedia article | 285k | 127k | −55% |
Accuracy
Both tools got the GitHub and Wikipedia answers right in all three runs. The Cloudflare Worker count came back between 172 and 179 from both tools, because the list mixes Workers and Pages projects. On the dev.to task, claude-in-chrome missed the reading-time label once and opencli missed it twice. One of opencli's misses ended early and cost $0.015, which flatters its number on that task. Without the dev.to task, opencli still cost 61% less.
What we take from it
- The gap is small on text-heavy pages like a GitHub pull request, where text and screenshots cost about the same.
- The gap is large wherever the model reaches for screenshots to find its way around.
- Every run carries about 95–120k tokens of session startup in both arms, so the difference in the browsing itself is larger than 42%.
Limits
Three runs per task is a small sample. We did not test a page where the data lives in a chart. A text read should do fine when a chart's numbers are in the page's HTML and badly when they are only drawn on a canvas. That is the next task we will add.
