AI & ML
Using profiles of cognitive capability to assess AI suitability for workplace tasks
arXiv cs.HC0 views
arXiv:2608.25623v2 Announce Type: replace-cross Abstract: Organisations deploying AI face a scoping problem: which tasks can be automated, which should remain with humans, and which are best shared between the two. Aggregate benchmark scores provide little insight into where systems will succeed or fail in practice, while human judgements of model capabilities quickly become outdated. We introduce a pipeline that profiles agents and tasks using a shared set of core cognitive capabilities. Cognitive capability profiling infers an agent's capabilities from performance on a benchmark battery annotated for the cognitive demands of each item. Task requirements weighting elicits from domain experts the relative importance of these same capabilities for their work. As both use a common set of cognitive dimensions, they can be updated independently as models and roles change, and combined to estimate AI suitability at the level of a domain, organisation, role, or individual duty. We validate capability recovery on synthetic agents, profile six AI systems, and elicit task requirements from 410 employees across six occupational domains. AI systems differed more across cognitive dimensions than across model families, while workplace activities converged on a shared cognitive core. The resulting scores provide a comparative scoping tool for identifying promising candidates for piloting and areas where current systems are unlikely to be well suited. We discuss extending the framework to profile human workers alongside AI systems, moving from AI suitability towards human-machine task allocation.
Source:https://arxiv.org/abs/2608.25623PreviousMetamaterial tactile displays with programmable virtual pixelsNextOnline Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training
Related posts
Auditing what an agent says it did, using its log
0dev.to Testing
Testing LIMS Software for Pharmaceutical Compliance: A Practical Guide
0dev.to Testing
Not every model makes a good attacker: OpenAI vs DeepSeek for red-teaming
0dev.to Testing
Testing AI meeting notes with one changed deadline and one missing owner
0dev.to Testing

Best ChatGPT Alternatives That Run Entirely on Your Own Device in 2026 (Free, No Account)
0dev.to OpenSource

Best Private Local AI App for Mac and Windows in 2026 (Runs on Your Computer and Remembers Your Work)
0dev.to OpenSource