Open Source
Large-Scale Study of GitHub Copilot Trajectories
arXiv cs.SE0 views
arXiv:2610.32313v1 Announce Type: new Abstract: AI coding agents are increasingly integrated into software development workflows. However, most coding agent studies rely on benchmarks similar to SWE-bench, which assume autonomous agents operating in a single trajectory. This assumption does not mirror real-world usage, where developers interactively collaborate with agents. We address this gap by studying human-in-the-loop agent trajectories in the wild, focusing on 7,581 Copilot-authored PRs with publicly accessible session data. We find that 58% of Copilot PRs have no human intervention, and that when developers do intervene, collaboration is brief (median of 2 Human-AI loops). Merged PRs also tend to have fewer initial trajectory iterations than unmerged ones. We also find that the trajectory following the first human intervention exhibits a significant drop of 20.8 iterations and 8.4 min in duration. However, trajectories following later interventions show minimal iteration drops and no significant duration change, signifying that most of the work happens in the initial trajectory. Our qualitative analysis of illustrative PRs explains this pattern: many developer interventions are small, focused requests that Copilot completes in shorter later trajectories. We also illustrate that developers collaborate with Copilot in multiple ways, including delegated work through feedback as well as adding their own commits. However, we find that developers may need to guide Copilot through multiple (sometimes repeated) interventions in the PR, or even across multiple editions of the PR.
Source:https://arxiv.org/abs/2610.32313PreviousPairing Matters: Performance, Failures, and Efficiency in Lightweight Coding AgentsNextFrom Faulty Claims to Incorrect Patches: An Empirical Study of Task-Grounded Reasoning in Coding Agents
Related posts
GIMP 3.2.8 released with security fixes
0dev.to OpenSource

We got real Linux on iPhone!
0dev.to Linux
EveryOS - A desktop-style web GUI for Linux servers.
0dev.to Docker
Show HN: HN Atlas – Discover tools to better engage with Hacker News
0Hacker News Show
Show HN: TextoCAD – Text to editable CAD (STEP/STL), runs locally with your key
0Hacker News Show
Owen Sound: From Transit Budgets to Ride Targets
0dev.to JavaScript