GitHub Agentic Workflows went generally available on October 28, after a public preview that started in June. You write the automation as a Markdown file, the gh-aw compiler turns it into a standard GitHub Actions workflow, and a coding agent runs it on your existing runners with read-only defaults. A week after the general availability announcement, a research team published the first large-scale archive of what those workflows actually do in production. The headline number: 178,151 of 562,877 archived execution logs, or 31.7%, are linked to runs that ended with a failure conclusion.
If GitHub infrastructure is now absorbing agent traffic at this scale, this dataset is the closest thing we have to a flight recorder. GitHub itself has been retooling for it, and I wrote about the storage side of that push when 5,000 agents pushed 7,200 times a second inside GitHub's Rubicon. This paper covers the workflow layer sitting on top.
The setup: agents you write in Markdown
A gh-aw workflow is a Markdown file in .github/workflows/ with YAML frontmatter. The frontmatter declares the trigger, permissions, allowed outputs, and any MCP servers. The prose below it is the task itself. Here is the example from the project's landing page:
--- on: daily safe-outputs: create-issue: --- Summarise yesterday's issues. Flag anything that needs a reply. The file compiles into ordinary Actions YAML, runs behind the Agent Workflow Firewall in a sandboxed container, and can be powered by GitHub Copilot CLI, Claude Code, Codex, or Antigravity. Proposed changes go through a safe-outputs process, and a separate threat detection job scans them before anything is applied.
What the researchers actually archived
The paper, GH-AWTrace (arXiv:2611.06866), was submitted on November 4. The authors found candidate workflows through GitHub code search using the comment every compiled file carries, This file was automatically generated by gh-aw, then verified each file actually had the generator header. That produced 25,014 verified workflows across 5,051 repositories.
Then they collected runs. 4,112,526 run records spanning August 14, 2025 to September 21, 2026, which is as far back as GitHub's 400-day run retention reaches. Logs are the harder race: public workflow logs are deleted after at most 90 days. For the 90-day window they grabbed 1,633,215 log archives, 56.1 GB of ZIPs. Of those, 1,070,338 turned out empty. Another 821,672 runs had no downloadable archive at all. What survived is 562,877 nonempty archives, belonging to 561,367 runs.
That survival filter matters for everything below. The dataset is what the retention window let them keep, not a clean sample.
The 31.7% number, read carefully
Of the 562,877 nonempty archives, 178,151 (31.7%) link to run records with a failure conclusion. The authors state the caveat themselves: this does not attribute every run failure to its agent. A run can fail because the runner died, because a provider hiccuped, or because the task was genuinely blocked. It is "a third of logged runs did not conclude clean", not "coding agents fail a third of the time".
For comparison with self-reported agent success stories, this is still a sobering baseline. When I covered Alibaba's measurement that the best coding agent finished 26% of end-to-end tasks, the pushback was that lab tasks are artificially hard. These are real repositories running their own automations, and a third of logged runs still failed.
What the failures look like in the logs
Two examples from the paper stand out, because neither is a model failure.
In one issue triage run, the agent called issue_read on the GitHub MCP server and got back McpError: filtered by integrity policy. The resource filter blocked the read, the agent could not triage what it could not see, and it reported the run as incomplete. The failure lived in the access policy, decided in frontmatter.
In a failure scanner run, the agent hit Authentication failed with provider on an internal endpoint, HTTP 403. The harness retried once, the second attempt failed the same way, and the job exited with code 1. Pure infrastructure.
The logs also capture adaptation. One agent changed its publishing command after a tool denial and continued. One review run dispatched three background reviewers, got two responses back, and used agreement between those two to select findings. The authors note there was no evidence of memory shared across runs, only delegation within a run.
Schedule triggers dominate the count, one trigger dominates the volume
Of the 6,040 workflows with logs, the median workflow produced 16 log archives in 90 days, with the middle half between 6 and 60. Schedule triggers are the common shape: 4,766 workflows (78.91%) run on a schedule, producing 252,691 logs (44.89%).
The outlier is pull_request_target. Fifty-nine workflows, 0.98% of the total, produced 73,594 logs, 13.07% of all log volume. That is roughly 1,247 executions per workflow versus about 53 for scheduled ones. issue_comment shows a similar shape at smaller scale, 171 workflows generating 58,312 logs. If you wire an agent to react to pull requests or comments, you are not writing a cron job, you are writing a service that runs whenever anyone types. Budget and permission limits on those triggers are not optional hygiene, they are the whole ballgame.
Permissions decide the output shape, not just the safety
My favorite case in the paper is a production workflow in Azure/azure-dev. Every week it reads feedback on merged pull requests and turns recurring reviewer requests into proposed updates to the repository's Copilot instructions. In August 2026 it examined 49 pull requests and proposed rules to reject ignored command-line flags and to redact credentials from URLs before printing them in logs.
Repository settings prevented this workflow from opening a pull request. So it saved a branch and filed an issue with the patch instead, and a maintainer refined and merged the change through pull request #9731. The permission configuration determined the delivery mechanism, and the useful work still landed. That is the pattern I want from agent automation: the config decides what the agent can touch, and the agent degrades gracefully when it is told no. It rhymes with what I found when Copilot's assisted permissions went GA: the interesting design surface is increasingly who or what decides yes.
Debug your own runs before the logs vanish
gh-aw ships its own inspection tooling, and it is worth learning before you need it:
gh aw logs my-workflow -c 10 --start-date -1w --firewall gh aw audit https://github.com/OWNER/REPO/actions/runs/12345678 --parse gh aw logs groups failed, timed-out, and cancelled runs into failure investigation clusters using failed jobs, MCP failures, missing tools, and blocked domains. The docs are refreshingly blunt about the limits: clusters describe shared symptoms, not proven root causes, they can overlap, and you should not sum their counts into a failure rate. gh aw audit breaks down a single run, including the behavior fingerprint, tool usage, MCP server status, and token and cost metrics.
And archive anything you might want later. Public logs disappear after 90 days. The entire reason this dataset exists is that somebody raced that clock.
Read the whole thing honestly
Independent review of the paper flags the weak spot: the 25,014 workflow denominator rests on a byte-exact header phrase plus a 384 KiB file-size cap in code search, with no cross-check against an independent enumeration. If the search index missed files, the headline counts are a sample of unknown recall, and the trigger distribution inherits that uncertainty.
The authors are upfront that the logs are partial observations rather than complete trajectories, and that run conclusions do not equal task completion. That last gap is the one I keep thinking about. A workflow run can conclude successfully while the agent quietly did nothing useful, and no dashboard will tell you. The dataset is on Hugging Face (jaykay9999/GHAWTrace) with a Python example that joins the tables and reproduces the 31.7% figure. If you run agentic workflows on real repositories, it is the best execution evidence available, warts and all.
