On 20 October Simon Willison published Battle Drones, a browser RTS in the Command & Conquer mould. He built it over 12 major versions and 321 commits, mostly with Claude Code, and he is candid about it: "I've reviewed almost none of the code." The repo is public, which makes it the best open dataset I have seen for a question every team adopting coding agents should ask: how much of our codebase did the agent write, and what happened to that code afterwards?

So I wrote a small, stdlib-only Python script that answers it from git log, cloned the repo and ran it. Below are the script, the real output and what I think it means for teams.

Why measure this at all

My team is small and agents now write a real share of our commits. "Is the agent productive?" is the wrong question. The useful ones are operational:

  • Review load. If most changed lines come from an agent, reviewers are the bottleneck. The cost of a merge is reading, not writing.
  • Churn. Code added and deleted within two weeks is work you paid for twice, and agents can produce a lot of it.
  • Fix-ups. If fix commits keep landing on files an agent just touched, that is where you need tests, a spec or a narrower prompt.
  • Hotspots. The files agents touch most are where a guardrail (a lint rule, a CODEOWNERS entry, a contract test) pays back fastest.

None of this needs a vendor dashboard. Agents already leave fingerprints in git.

How it detects agent commits

Claude Code adds a Co-Authored-By: Claude ... <noreply@anthropic.com> trailer, and Claude Code on the web also adds a Claude-Session: line with a session URL. Copilot's coding agent commits as copilot-swe-agent[bot] or a +Copilot@users.noreply.github.com address. Cursor, Codex, Devin and Aider have their own patterns. The script matches one regex per agent against author, committer and the full message. Everything else counts as human. Merge commits and CI bots are skipped.

Then, in one pass over the full patch history (git log -p with zero context lines), oldest commit first, it computes:

  • Share: commits and changed lines (added plus deleted) per class, ignoring generated paths such as dist/, lockfiles and minified bundles.
  • Churn: an added line counts as churned if a later commit deletes an identical line in the same file within N days (default 14). This is content matching, not true blame, so treat it as an estimate.
  • Fix-ups: commits whose subject says fix, bug, revert, regression and so on, split by whether they touch a file that agent or human work changed inside the window.
  • Hotspots and timeline: top files by agent-changed lines, plus a per-day table.

The script

#!/usr/bin/env python3 """agentaudit.py: how much of a git repo did coding agents write? Usage: python3 agentaudit.py [REPO] [--days 14] [--top 8] (stdlib only) """ import argparse, collections, datetime as dt, re, subprocess # One regex per agent, matched against "author / committer / full message". AGENTS = { "Claude": r"noreply@anthropic\.com|co-authored-by:[^\n]*claude|claude-session:" r"|claude\.ai/code/session|generated with \[claude code\]", "Codex": r"co-authored-by:[^\n]*codex|chatgpt-codex-connector|codex@openai\.com", "Copilot": r"copilot-swe-agent|\+copilot@users\.noreply|co-authored-by:[^\n]*copilot", "Cursor": r"cursoragent@cursor\.com|co-authored-by:[^\n]*cursor", "Devin": r"devin-ai-integration", "Aider": r"\(aider\)|co-authored-by:[^\n]*aider", } AGENTS = {k: re.compile(v, re.I | re.M) for k, v in AGENTS.items()} BOT = re.compile(r"\[bot\]|actions@users\.noreply|github-actions", re.I) # CI, not agents FIX = re.compile(r"^(revert|fixup!|hotfix)|\b(fix(es|ed)?|bug|regression|broke|broken|oops)\b", re.I) TRAILER = re.compile(r"^co-authored-by:\s*([^<\n]+?)\s*<", re.I | re.M) # Generated files inflate line counts; skip them (override with --skip). SKIP = r"(^|/)(dist|build|vendor|node_modules|docs/assets)/|\.lock$|-lock\.json$|\.min\.|\.map$" FMT = "%x00%H%x1f%an <%ae>%x1f%cn <%ce>%x1f%at%x1f%P%x1f%B%x1e" def who(author, committer, msg): text = f"{author}\n{committer}\n{msg}" return next((name for name, rx in AGENTS.items() if rx.search(text)), None) def audit(repo, days, skip_rx): skip_rx = re.compile(skip_rx) out = subprocess.run(["git", "-C", repo, "log", "--reverse", "-p", "-M", "--unified=0", "--no-color", "--format=" + FMT], capture_output=True, text=True, errors="replace", check=True).stdout window = days * 86400 s = collections.defaultdict(collections.Counter) # stat name -> {"agent"/"human": n}  per_agent, models = collections.Counter(), collections.Counter() files, day = collections.defaultdict(collections.Counter), collections.defaultdict(collections.Counter) added = collections.defaultdict(lambda: collections.defaultdict(collections.deque)) last_touch = {} # path -> (ts, cls)  for chunk in out.split("\x00")[1:]: head, patch = chunk.split("\x1e", 1) sha, author, committer, ts, parents, msg = head.split("\x1f") ts = int(ts) if len(parents.split()) > 1: s["merges"]["all"] += 1 continue agent = who(author, committer, msg) if not agent and BOT.search(author): s["merges"]["bots"] += 1 continue cls = "agent" if agent else "human" if agent: per_agent[agent] += 1 models.update(t for t in TRAILER.findall(msg) if AGENTS[agent].search(f"co-authored-by: {t}")) s["commits"][cls] += 1 if not agent and re.search(r"\(#\d+\)$", msg.split("\n", 1)[0].strip()): s["squash"]["human"] += 1 # squash-merged PR: author unknown  d = dt.datetime.fromtimestamp(ts, dt.timezone.utc).strftime("%Y-%m-%d") day[d][cls] += 1 path, in_hunk, skip, touched = None, False, False, set() for line in patch.split("\n"): if line.startswith("diff --git "): m = re.match(r"diff --git a/(.*) b/(.*)", line) old, path = (m.group(1), m.group(2)) if m else (None, line) if old and old != path and old in added: # follow renames  added[path] = added.pop(old) skip = bool(skip_rx.search(path)) if not skip: touched.add(path) in_hunk = False elif line.startswith("@@"): in_hunk = not skip elif in_hunk and line[:1] in ("+", "-"): content = line[1:].strip() s["lines"][cls] += 1 files[path][cls] += 1 day[d][cls + "_lines"] += 1 if line[0] == "+": s["added"][cls] += 1 if len(content) > 2: # skip "}", "", ");"  added[path][content].append((ts, cls)) elif added[path][content]: t0, c0 = added[path][content].popleft() # oldest matching add  if ts - t0 <= window: s["churn"][c0] += 1 if FIX.search(msg.split("\n", 1)[0]): s["fixes"][cls] += 1 hit = {last_touch[p][1] for p in touched if p in last_touch and ts - last_touch[p][0] <= window} s["fix_after"].update(hit) for p in touched: last_touch[p] = (ts, cls) return s, per_agent, models, files, day def pct(a, b): return f"{100 * a / b:5.1f}%" if b else " n/a" def report(repo, days, top, skip_rx): s, per_agent, models, files, day = audit(repo, days, skip_rx) c, l = s["commits"], s["lines"] nc, nl = sum(c.values()), sum(l.values()) print(f"repo: {repo} window: {days} days merges skipped: {s['merges']['all']} bot commits skipped: {s['merges']['bots']}") print(f"commits agent {c['agent']:>6} {pct(c['agent'], nc)} human {c['human']:>6} {pct(c['human'], nc)}") print(f" human commits that look like squash-merged PRs (true author unknown): " f"{s['squash']['human']}") print(f"lines agent {l['agent']:>6} {pct(l['agent'], nl)} human {l['human']:>6} {pct(l['human'], nl)}") print("agents " + ", ".join(f"{k} {v}" for k, v in per_agent.most_common())) print("models " + ", ".join(f"{k} {v}" for k, v in models.most_common(5))) for cls in ("agent", "human"): print(f"churn {cls}: {s['churn'][cls]} of {s['added'][cls]} added lines deleted within " f"{days}d ({pct(s['churn'][cls], s['added'][cls]).strip()})") print(f"fixes fix/revert commits: {sum(s['fixes'].values())} (by agent {s['fixes']['agent']}, " f"by human {s['fixes']['human']})") for cls in ("agent", "human"): print(f" touching files last changed by {cls} work <= {days}d earlier: " f"{s['fix_after'][cls]} ({s['fix_after'][cls] / max(c[cls], 1) * 100:.1f} per 100 {cls} commits)") print(f"\ntop {top} files by agent-changed lines:") for p, n in sorted(files.items(), key=lambda kv: -kv[1]["agent"])[:top]: print(f" {n['agent']:>6} {pct(n['agent'], n['agent'] + n['human'])} agent {p}") print("\nday commits A/H lines A/H agent share of lines") for d in sorted(day): n = day[d] share = n["agent_lines"] / max(n["agent_lines"] + n["human_lines"], 1) print(f"{d} {n['agent']:>4}/{n['human']:<4} {n['agent_lines']:>7}/{n['human_lines']:<6} " f"{'#' * round(share * 20):<20}") if __name__ == "__main__": ap = argparse.ArgumentParser(description=__doc__) ap.add_argument("repo", nargs="?", default=".") ap.add_argument("--days", type=int, default=Source:https://dev.to/neeraj_sharma_757fbb49aa7/98-of-battle-drones-was-written-by-claude-i-audited-334-commits-with-140-lines-of-python-2jmf

Related posts