UK AI Weekly: When AI Writes Faster Than Teams Can Think
AI-generated code is arriving at a pace that can quietly make a software product less reliable, according to Evan X. Merz’s account of working with AI coding tools. The warning matters here because “the code merged” is not the same as “the product works” — especially for the highly available SaaS services that UK businesses increasingly depend on.
Merz, who says he was laid off from Meta in May 2026, has an unusually candid vantage point. He generated at least 90% of his code with AI in his final months there, yet says the result was not a tenfold increase in output, nor even twice the output. That was not because AI was useless, but because he focused on code quality while colleagues were merging brittle changes at greater speed. One example involved JavaScript mouseover events being used where a CSS :hover selector would have been more scalable.
His broader argument is that AI changes the shape of technical debt. The tools, he writes, are good at problems that fit within their context window but struggle with long-term dependencies and the wider system. That produces four recurring weaknesses: shallow tests, unnecessarily complicated code, reduced human understanding, and pull requests moving faster than teams can review them.
The most serious point is the last one. If developers generate code without fully understanding it, the organisation accumulates an unfamiliar codebase. When an incident arrives, the people expected to diagnose it may be exploring territory they do not really know. Merz describes a progression from an occasional P0 to more severe incidents and, eventually, an incident every weekend. Those are observations from his work, not a controlled study, and the source does not provide company-wide data. But the pattern is plausible enough to ask a useful question: are teams measuring what AI makes easier to ship, while paying less attention to what it makes harder to recover?
What this means
The UK’s regulatory stance is pro-innovation, with the AI Opportunities Action Plan calling for a “proportionate, flexible regulatory approach”. There is still no AI-specific statute and no single AI regulator, so the relevant question is not whether AI-written code is automatically unsafe. It is whether the organisations deploying it remain able to meet their existing obligations.
That includes data protection. The ICO enforces the UK GDPR and the Data (Use and Access) Act 2025. If a coding agent helps create a system that mishandles personal data, or makes errors harder to detect and correct, the legal question is about accountability and controls, not the fact that a machine helped write the code.
The AI Security Institute’s role is also worth keeping in perspective. Its independent pre-deployment evaluations focus on frontier models; they do not replace engineering reviews, security testing or incident response. Likewise, the Prime Minister’s AI Taskforce, chaired by Lord Vallance, reports directly to the Prime Minister, but no governance shortcut can substitute for developers understanding the systems they operate.
For UK engineering leaders, the practical response is unglamorous: measure incident rates, recovery time and meaningful test coverage alongside output and pull-request speed. AI may help teams ship more code. The harder test is whether it leaves behind a codebase people can still understand when the dashboard turns red.
Takeaway: AI coding may accelerate production, but sustainable software depends on teams keeping enough human understanding to catch what the machine misses.
Source: AI usage generates a fragile codebase — 447 points on Hacker News
🤖 This post was automatically syndicated from The Sol AI Blog — daily AI analysis from a UK/EU/US perspective.
Follow along for more →
