Before trusting a weight edit, I measured the bug it fixes

Krishanu Das · November 2026 · Code, plans and raw records: github.com/Krishanu-das-05/weight-space-training

TL;DR. A recent post showed an AI agent editing the weights of Qwen3.5-0.8B and raising its MATH-500 score from 7.9% to 48.1%. I tried to check the edit on a laptop GPU, and along the way found that "the model finished" and "the model was right" have to be measured separately. With thinking on, the unedited model finished naturally on only 2 of 30 GSM8K questions at an 8,192-token budget. Giving it more tokens did not help. With the published edit applied, 10 of 30 finished and 9 of 30 were correct (vs. 1 of 30). That matches a prediction I made in advance from the authors' own logs. The paired difference has a raw p of 0.02, which does not survive the multiple-comparison correction I committed to beforehand (Holm p = 0.11). Small sample, one model, local hardware: read it as a careful replication attempt, not a benchmark.

Background

Weight-space programming (Takehi et al., 2026) asks whether an LLM agent can read and write neural network weights directly, without training. One of its headline results is an edit to Qwen3.5-0.8B in thinking mode. The edit changes two MLP down_proj matrices (layers 15 and 19), and the post reports MATH-500 rising from 7.9% to 48.1%; the authors' published results table also shows GSM8K rising from 13.8% to 67.1%. Both were evaluated with up to 32,768 generated tokens.

I wanted to answer two questions on my own machine:

1. Does the unedited model fail in the way the edit is supposed to fix?
2. Does the published edit change that, under a budget a laptop can afford?

Common setup for everything below:

• Model: Qwen/Qwen3.5-0.8B, pinned revision, BF16, Hugging Face Transformers, an RTX 2000 Ada laptop GPU, batch size 1.
• Sampling: temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5. These match the authors' evaluation settings.
• Grader: strict. The answer must be an integer inside \boxed{} after </think>, and the response must end naturally (EOS). A box inside unfinished reasoning does not count.
• Determinism: fixed seeds, deterministic CUDA algorithms, no time-based stopping. Every run has a token budget and a written plan, fixed before any generation.

Experiment 1: Replaying the published edit

Does the edit help on a laptop-sized budget?

• Setup: two handmade math problems, fixed seeds, a 512-token cap. The edit is applied in memory, which changes 130,188 BF16 values, and its tensor hashes are verified. The originals are restored and re-verified afterwards.
• Result: both the original and the edited model hit the cap on both problems without producing an answer. At 2,048 tokens on the first problem, both hit the cap again.

This told me nothing about the edit and a lot about my setup: I couldn't evaluate a fix until I knew where and why the base model failed.

Experiment 2: Is it the prompt?

Does asking for step-by-step reasoning cause the poor completion?

• Setup: original weights only. Thinking on or off, crossed with two instructions: "reason step by step" or "give only your final answer". Two problems, two seed groups, 16 responses, a 1,024-token cap.

With thinking off, the final-only instruction raised completion from 2/4 to 4/4 and cut output by 83%. Correctness stayed at 2/4. The two shortest replies were both wrong: \boxed{264} in 10 tokens (expected 1048) and \boxed{70.4} in 11 tokens (expected 66). The two correct "final-only" replies ignored the instruction and explained their working.

Lesson: completion and correctness are different measurements. A model that stops sooner is not a model that answers better.

Experiment 3: Is it the token budget?

If thinking mode just needs more room, accuracy should climb as the budget grows. I committed to that hypothesis, with a numeric threshold, before running.

• Setup: 30 questions sampled with a fixed seed from the GSM8K train split (pinned dataset commit and SHA-256). Thinking on vs. off, same step-by-step instruction, one seed per question shared by both conditions.
• One generation, five budgets: each response is generated once up to 8,192 tokens and then scored as if the cap had been 512, 1,024, 2,048, 4,096 or 8,192. Because generation is causal and deterministic, the first k tokens of an 8k run are exactly what a k-token run would produce. I checked this with real short runs, comparing token IDs, before trusting it.

The hypothesis was rejected. At 8,192 tokens, only 2/30 thinking-on responses ended naturally, against 28/30 with thinking off. More budget didn't help, because thinking mode wasn't finishing at all.

A check against the authors' own logs. The post's evaluation logs are public. Most of their unedited thinking-on responses never close their thinking. In one sample, the model reaches the correct answer, 130, then writes "Wait, I'll write \boxed{130}. Okay." over and over until it hits the 32k cap. The edited model answers the same question and stops. Estimating token counts from text length, only about 9% of their unedited GSM8K responses close their thinking within 8k tokens, and about 10% even at 32k. My 2/30 (6.7%) is consistent with that. The base model's problem is not a short budget; it doesn't stop.

Experiment 4: Does the edit fix it?

Same 30 questions, same seeds, same 8,192-token cap, with the published edit applied.

Before running, I wrote down a prediction from the authors' logs: about 30% (range 28–40%) of edited responses should close their thinking within 8k tokens, against about 9% unedited. Natural completion was declared the primary outcome, and correct delivery the secondary one.

• The prediction held: thinking closed in 36.7% of edited responses, inside the predicted range.
• Paired comparison: on natural completion, the edit won on 9 questions, the original on 1, and 20 were ties. Exact McNemar p = 0.021. After the Holm correction across the five budgets I had committed to, p = 0.107, so by my own pre-registered standard this is suggestive, not significant.
• One failure I had to handle: on question 12, the edited model produced 8,192 tokens that decoded to 669,118 characters, a degenerate run-on that tripped a sanity bound in the grader and crashed the run. I added a rule before generating the remaining 19 questions: keep the raw output, score any out-of-bound response as incorrect, and decide completion from token IDs. The rule is disclosed in the report, and it changes none of the 11 results already saved.

Putting it together: at my budget, the edit makes thinking mode finish about five times as often. The authors' logs suggest the gap widens at 32k, which is the budget their headline numbers use. On these 30 questions, though, simply turning thinking off (16/30 correct in Experiment 3) still beat the edited model with thinking on (9/30). They are different runs on the same questions and seeds, so treat that as a note, not a test.

How I kept it honest

• Plans frozen before running: each experiment had a plan file (questions, seeds, sampling, grader, statistics, stopping rules) with source-file hashes, and nothing ran until the plan was approved.
• Budgets and stops: token budgets reserved before each call, no wall-time cutoffs, records saved after every response so a crash or a sleeping laptop loses nothing.
• Guards that held: when my first development pool turned out to have 4 questions instead of 30, the pipeline refused to start. The run stopped at question 12 instead of silently scoring a broken output. Every resume had to re-apply the edit and verify its hashes before generating anything.
• The grader never changed after results were seen, and the held-out question set was never used.

Most of the useful moments in this project were the safeguards stopping the work exactly where they were supposed to.

Limitations

• Small samples: 30 questions, one seed each, so only large effects are detectable. Experiments 1 and 2 used 2 handmade questions.
• Likely training-data overlap: GSM8K train is likely in Qwen's training data. The absolute accuracies here are not capability claims.
• Shorter budget: I tested at up to 8,192 tokens; the post evaluates at 32,768. My results don't contradict theirs; they describe a cheaper setting.
• Strict grading: integer \boxed{} answers after a closed </think> and a natural EOS only.
• Log-based estimates are approximate: the estimates from the authors' logs use text length, not token counts.

What's next

• Compare base and edited models at 16k and 32k tokens, where the logs suggest the effect is larger.
• Use more questions and seeds, so a difference of this size can clear the correction.
• Classify the non-finishing responses (loops vs. long but progressing) without new inference.

Reproduce it

Code, plans, records and notebooks: github.com/Krishanu-das-05/weight-space-training. Each experiment has a report and a notebook that recomputes every table from the raw records without loading a model. The full test suite runs offline.

Credits

The weight edit and evaluation logs are from Rikiya Takehi et al., RikiyaT/lm_understand-logs (CC BY 4.0), used unmodified at a pinned commit. Qwen3.5-0.8B is Apache-2.0. GSM8K is MIT. Any mistakes here are mine.
![ ](https://dev-to-uploads.s3.us-east-