Result first: an agent made 16 commits, and somewhere along the way add 2 3 started returning -1. I gave git bisect one good commit, one bad commit and a 3-line test script. It ran the test 4 times and named the first bad commit, "commit 11", which is exactly the one where the bug was planted.

printf '#!/usr/bin/env bash\nsource ./calc.sh\n[ "$(add 2 3)" = 5 ]\n' > ../test.sh && chmod +x ../test.sh git bisect start HEAD <known-good-commit> git bisect run ../test.sh git bisect reset # go back to where you started

The rules for the test script (from the git docs): it "should exit with code 0 if the current source code is good/old, and exit with a code between 1 and 127 (inclusive), except 125, if the current source code is bad/new." Exit code 125 means "this commit can't be tested, skip it." "Any other exit code will abort the bisect process."

Why this suits agent workflows:

  • Agents make many small commits, and bisect is a binary search, so the number of test runs grows slowly as the history grows (here 4 runs for 16 commits).
  • Keep the test script outside the repo or untracked, as ../test.sh is here, so checking out old commits doesn't change the test itself.
  • Use exit 125 for commits that don't build, rather than marking them bad.

Do this now: next time an agent session ends with a regression, write the one-line check first, then run git bisect run before reading any diffs.

Sources

From Penloom Studio's free AI lessons: penloomstudio.com