Hold OSS Model Notes Behind a Local Review Contract

An OSS model note stays non-binding until a local contract passes. The contract locks one command, one diff scope, and two exit codes. A free model may comment only after that gate.

Why unbound notes fail

A long model reply can sound like a finished review. It still lacks a rerun of the failing command. Maintainers then merge model confidence instead of local evidence.

This workflow splits the work into three lanes. Local reproduction owns the claim about the bug. A disposable rerun and a model read stay secondary.

Contract fields that matter

Keep the contract in review-contract.json next to the patch. Record only facts that a stranger can check. Leave coding style opinions out of that file.

{ "repo": "example/widget", "base": "origin/main", "command": "python -m unittest tests.test_widget", "expect_before": 1, "expect_after": 0, "allowed_paths": ["widget/parse.py", "tests/test_widget.py"], "model_role": "non_binding_reader" } 

The expect_before field stores the exit before the patch. The expect_after field stores the exit after the patch. Both numbers must come from commands, not from chat.

The widget paths and example.invalid host are placeholders, not a real project. Swap in the public repository before a real review run.

Three lanes, one owner

Use the free server as a clean rerun host. Use free model access as a narrow second reader.

The operator presents MonkeyCode as an open-source project. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Those claims cover the rerun lane and the read lane only.

This draft does not name models, quotas, hardware, or duration. Cite a current primary source before publishing any of those figures.

Step 1. Freeze one failing command

Choose a command that fails on the base commit today. Write it into the contract before any edit. Ban prompts, watches, and commands that never exit.

git fetch origin git switch --detach origin/main python -m unittest tests.test_widget echo "before=$?" 

Copy the printed exit code into the expect_before field. A zero means this command does not show the bug. Replace that command before writing any code patch.

Run the snippet without errexit, or the shell stops early. A stopped shell never prints the before marker. The missing marker is a failed reproduction, not a pass.

Step 2. Limit the patch to named paths

Open a detached worktree for the patch experiment. Check the diff, then apply it only if the path list matches. A wider diff makes the written contract false.

git worktree add --detach ../review-wt origin/main git -C ../review-wt apply --check patch.diff git -C ../review-wt apply patch.diff git -C ../review-wt diff --name-only > changed.txt 

Compare changed.txt with allowed_paths before the next lane. Extra paths mean the contract is already stale. Update the list or remove the extra file.

Step 3. Capture the local after log

Run the locked command inside the review worktree. Save the combined log and the exit marker. Never trim a failure just to save space.

cd ../review-wt python -m unittest tests.test_widget > after.log 2>&1 echo "after=$?" | tee -a after.log 

Require the after marker when expect_after equals zero. A different marker blocks every later review lane. The local log stays the primary review evidence.

Step 4. Confirm on a disposable server

Push nothing private into the disposable server session. Clone the public base, then apply the same patch file. Run the same command and record the server exit beside the local one.

Reuse the same patch.diff on the server lane. Do not rebuild the patch from memory on that host.

# Proposal only. Replace the placeholder before any real run. git clone --depth 1 https://example.invalid/widget.git review-srv cd review-srv git apply --check ../patch.diff git apply ../patch.diff python -m unittest tests.test_widget echo "server=$?" 

Label the server block as an unexecuted proposal. A free server may lack compilers, tokens, or long job time. Record that gap and keep the local log as authority.

Step 5. Build a redacted read packet

Copy four items into a folder named packet. Include the contract, the path list, both exit lines, and a scoped diff. Strip secrets, env files, and absolute home paths first.

mkdir -p packet cp review-contract.json changed.txt packet/ git -C ../review-wt diff -- widget/parse.py tests/test_widget.py > packet/scope.diff python3 -c 'from pathlib import Path; text = Path("after.log").read_text(encoding="utf-8", errors="replace"); lines = [ln for ln in text.splitlines() if ln.startswith("after=")]; Path("packet/exits.txt").write_text("\n".join(lines) + "\n", encoding="utf-8")' 

That packet is the only model input. A full repository upload adds risk without review value. Delete the packet after the note is written.

Step 6. Request a non-binding read

Ask the free model to list uncovered risks. Require each bullet to cite a path or a contract key. Forbid a verdict of correct, safe, or ready.

Role: non_binding_reader Files: review-contract.json, changed.txt, exits.txt, scope.diff Task: list risks the locked command does not cover Rule: cite a path or a contract key in every bullet Rule: do not approve the patch 

A free model session can serve this narrow read. The model can still miss hidden fixtures and local conventions. A human maintainer still decides whether the patch lands.

Step 7. Check the contract before the note is posted

Run the checker on the contract, the path list, and the local log. Post model bullets only when the checker prints success. Prefix each bullet with the non-binding label.

python3 check_review_contract.py review-contract.json changed.txt after.log 

The script below is a local unexecuted proposal. It checks fields, role, path scope, and the exit marker. It never contacts a network and it never opens a pull request.

#!/usr/bin/env python3 """Proposal: validate an OSS review contract before model notes.""" import json import sys def main() -> int: if len(sys.argv) != 4: print("usage: check_review_contract.py CONTRACT CHANGED LOG") return 2 contract_path, changed_path, log_path = sys.argv[1:4] contract = json.loads(open(contract_path, encoding="utf-8").read()) required = [ "command", "expect_before", "expect_after", "allowed_paths", "model_role", ] missing = [key for key in required if key not in contract] if missing: print("missing fields:", ", ".join(missing)) return 2 if contract["model_role"] != "non_binding_reader": print("model_role must stay non_binding_reader") return 2 changed = [ line.strip() for line in open(changed_path, encoding="utf-8") if line.strip() ] extra = sorted(set(changed) - set(contract["allowed_paths"])) if extra: print("paths outside contract:", ", ".join(extra)) return 3 log = open(log_path, encoding="utf-8").read() marker = "after=%s" % contract["expect_after"] if marker not in log: print("local log lacks", marker) return 4 print("contract ok; model notes remain non-binding") return 0 if __name__ == "__main__": raise SystemExit(main()) 

Step 8. File the maintainer note

Use a fixed note shape so reviewers can scan it. Put local evidence first and model text last. Keep the model section short enough to scan.

Command: python -m unittest tests.test_widget Local before: 1 Local after: 0 Server after: not run Checker: contract ok Model role: non-binding Model notes: - non-binding: tests/test_widget.py covers only the named parser path 

A missing server exit line is still acceptable. A missing local exit line is not acceptable. Readers should rerun the command before they trust the note.

How failed checks should end

Exit 2 means required fields or the model role are wrong. Exit 3 means the diff left the allowed path list. Exit 4 means the local log lacks the expected marker.

Repair the failed lane before another model call. Do not ask a second model to overwrite a bad log. Edit the contract only when the command itself changed.

Limits that stay in force

One command can pass while a neighbor behavior fails. A clean server can still miss private fixtures or system packages. Free model wording can be fluent and still incomplete.

A log line that merely contains the marker can fool this check. Treat the checker as a gate, not as full test proof. Read the unittest summary before accepting the marker.

Normalize relative paths before running the set comparison. A leading prefix can cause a false extra-path failure.

This article states no quota, latency, or uptime figure. Those values were not verified against a primary source here. A free tier is optional assistance, not a promised production host.

Who should not use it

Skip the flow when the patch holds secrets or customer data. Skip it when project rules ban external model reads. Skip it when the bug depends on devices the server cannot attach.

Skip it for broad refactors that no single command represents. Skip it when the review must be a formal security audit. A non-binding model reader cannot replace that audit.

Practice to keep

Store the contract, the log marker, and the checker result with the patch. Mark every model sentence as non-binding in the review thread. Ask one maintainer to rerun the locked command locally.

Optional reruns and model reads can support that second look. They do not replace the local exit code. Operators who keep evidence separate from commentary can try the optional lanes in MonkeyCode.