Yesterday I found that trimming issue text to its first 300 characters nearly doubled a small model's accuracy: 17 out of 24 tasks solved, against 9 out of 24 when the model saw the full text. The failure mode was obvious. The model fixated on issue 412, a crash-on-launch bug, and answered 412 for tasks where the answer was something else entirely. A reader, nikodim_apps, asked the natural next question: what if you put a short summary of every issue first, and the full text after it? The model would see the whole field before it locks onto one candidate. That sounded right. I ran it. It did not work, and the way it did not work is more interesting than a plain loss.

The setup is identical to yesterday's, so the numbers compare cleanly. Same model, Qwen2.5-1.5B-Instruct, quantized, running locally. Same 12 tasks, same prompts, temperature 0, six calls per task at most. Three arms: full text of all five issues, trimmed text (first 300 characters plus title and label), and the new one — a five-field summary of every issue placed first, then the full text. A thirteenth task, suggested by raju_dandigam, hides its decisive clue after the first 300 characters. The trim arm cannot see it; the summary and full arms can. Every arm ran twice; the second run came back bit-identical to the first, temperature 0 plus the server's prompt cache, so the two reps are one measurement repeated, not two independent ones. I am counting them as 24 tasks, not 48.

The numbers

Summary-first solved 16 of 24 (66.7%). Trim solved 17 of 24 (70.8%). Full solved 9 of 24 (37.5%). Summary-first is a huge improvement over full text and a dead heat with trim, and it costs 2.6 times trim's tokens to get there. Mean prompt tokens per turn: summary 1,724, trim 673, full 1,421. Per task: summary 5,172, trim 2,020, full 5,475. The summary arm's first prompt alone is 7,686 characters against trim's 2,275. Every task in every arm took exactly three turns with exactly one second look: the model lists the issues, re-reads the one it is about to answer, and answers. That second look is a confirmation, not a search.

So the summary buys nothing trim does not already buy, at two and a half times the price. That is the headline. But the four problems below are why it matters, because each one is a different way the obvious fix failed.

1. The fixation survives the summary

The summary arm answered issue 412 on 10 of its 24 tasks. Eight of those answers were wrong. Every single one of its eight failures is the same answer: 412, bug. Tasks whose correct answers were 397 (enhancement), 393 (documentation), 404 (bug), and 375 (good first issue) all came back 412.

Think about what that means. The summary of 412 is sitting right there in the first prompt, next to four other summaries, and the model still locks onto it. The summaries did not give the model a map of the field; they gave it five candidates, and it picked the loudest one — the crash-on-launch bug, the most dramatic issue in the set — and stopped looking. A summary does not fix a fixation. It just makes the thing the model fixates on arrive sooner.

2. The summary costs more than the full text's problem, not less

Trimming works because it removes the text that drags the model toward the wrong issue. Summarizing does the opposite. It adds text on top of the full text. The summary arm's prompt is bigger than the full arm's, and its token bill per task (5,172) is nearly the same as full (5,475) — while trim does the same job for 2,020. If you are paying per token, summary-first is the worst of both worlds: full-text accuracy would be embarrassing at that price, and trim-level accuracy is available for less than half.

3. The late-clue test came back mixed, and one arm's win is a leak

Task 13 hides its decisive clue after the first 300 characters. Summary solved it twice, trim solved it twice, full failed twice. On the surface that looks like a win for summaries. It is not, quite.

The full arm re-read issue 412 and answered 412. The fixation again, now on a task designed to punish it. The trim arm, though, did not solve it from the clue — it could not see the clue. It solved it from the visible fields alone: issue 417's title and its label, "bug", against the distractor 419's "enhancement". The label field leaks the signal the late clue was meant to isolate. So trim's 2-for-2 on this task is real accuracy and a flawed test at the same time. The honest reading: the late clue did not decide anything, because the labels decided it first. If you want a clean test of whether a model reads past 300 characters, strip the labels too.

4. The second rep added nothing

Both reps returned identical answers and identical token counts in every arm. That is expected at temperature 0 with a prompt cache, and it is worth saying plainly because it changes what "two reps" means here. I have one measurement, confirmed stable, not two measurements. The wall clock, meanwhile, told a different story — the summary arm's first rep took 2,914 seconds under the morning's CPU load and the second took 269 — but token counts do not care about wall time, and neither should the conclusion.

What I would do instead

Trimming stays the default. It is cheaper, it scores the same, and it has one clear mechanism: less text, fewer chances to fixate. Summaries are not useless — they are the right shape when the model genuinely cannot see the whole field, when the candidate set is large and the summaries are the only way in. But when the field is five issues and the model can already see them, a summary is decoration on a fixation. The next experiment is not a better summary. It is a prompt that makes the model name two candidates before it answers, or a test setup that hides the labels, or both. The fixation is the problem. Everything else is formatting.

Have you watched a small model fixate like this, one candidate it keeps answering no matter what you show it first? What broke the fixation for you: a different prompt, a different setup, or just a bigger model? I am curious what worked, because summaries did not.

P.S. Everything I write stays free — that part doesn't change. But if this one was worth something to you, you can buy me a coffee ☕ (one-time or monthly, both work). It covers the API credits and hosting behind these experiments — and honestly, it's also the nudge that gets the next one started on the evenings I'd otherwise skip it. — Salam

Buy me a coffee

AheadKit on Gumroad — Beginner-Friendly Books & Packs