AI & ML
Beyond Correctness: How Novice Software Engineering Students Exercise Oversight of Generative AI
arXiv cs.SE0 views
arXiv:2610.32118v1 Announce Type: new Abstract: Generative AI (GenAI) is shifting software engineering students' work from producing artefacts towards evaluating and supervising AI-generated outputs. However, little is known about what characteristics students identify in outputs when reporting interventions, or how they exercise oversight during authentic software engineering work. We analysed 962 student-AI interaction turns documented by 145 first-year software engineering students through structured AI Journals, including 172 student-declared interventions. Using a multi-dimensional coding framework, we characterised the output characteristics identified in these reported interventions, students' reasons for intervening, the actions they took, and how those actions were enacted. Fitness problems were identified in 74.4% of interventions, compared with 16.9% involving correctness problems, indicating that oversight more often concerned whether plausible AI outputs were suitable for the engineering task than whether they contained explicit errors. Among classifiable interventions, students most frequently modified generated outputs (51.0%) or refined their prompts (28.2%), with both response types commonly associated with engineering fit. Within-student analysis further showed that outputs perceived as containing minor flaws were associated with lower evaluation confidence than outputs perceived as perfect. Taken together, these findings characterise novice AI oversight as a process that extends beyond error detection to encompass both judgement of fitness for purpose and active intervention in AI-assisted engineering work.
Source:https://arxiv.org/abs/2610.32118PreviousCrowdStrike Channel File 291: An Engineering Analysis of a Global Endpoint OutageNextPairing Matters: Performance, Failures, and Efficiency in Lightweight Coding Agents
Related posts
What tech entrepreneurs can teach scientists about failure
0Nature News

X for iPhone and iPad just added over a dozen new app icon options
09to5Mac
iOS 27.2’s clever new Messages feature can be tailored to your needs
09to5Mac

OpenAI is retiring ChatGPT Classic app, will soon combine Chat and Work tabs in new app
09to5Mac
Cilium 1.20.0 Requires BTF Flag in Linux Kernel, Raspberry Pi OS Kernel Incompatible Without It
0dev.to
YC Labs and Olam Labs ran 50 AI agents in a Minecraft civilization experiment
0dev.to