AI & ML
CanvasObs: Structured Spatial Observation for Agent Editing on 2D Infinite Canvases
arXiv cs.HC0 views
arXiv:2610.27176v1 Announce Type: new Abstract: Editing agents on 2D infinite canvases must reason about application state that pixels may not reveal. Targets may be offscreen or covered, while visual containment and touching arrows may disagree with recorded frame membership or connector bindings. We ask how a structured observation layer can support accurate object selection during agentic editing on 2D infinite canvases. We present CanvasObs, which augments visual context with a structured payload and guidance for reading four components: Identity & State, Visibility & Structure, Relations, and Offscreen Object bounds in action coordinates. A Remote Team Handoff case study illustrates retrieving an editable offscreen copy, choosing a covered chart by color, separating cards by frame membership, and preserving an evidence connection. We turn these cases into ten controlled configurations and compare CanvasObs with the baseline agent in 1,200 browser trials across three models. CanvasObs improves Correct target for every model. Combined across models, it raises Correct target from 16.5% to 48.3%, an increase of 31.8 percentage points (95% confidence interval [29.0, 34.7]), and Full task success from 4.3% to 25.0%. These results show that compact structured observation can improve object choice on 2D infinite canvases without redesigning the editing pipeline.
Source:https://arxiv.org/abs/2610.27176PreviousToo Long to Watch, Too Short to Leave: Designing for Agent Micro-WaitsNext"It Wouldn't Have Been So Apparent to Me": Evaluating How Thought-Level Progress Tracking Supports Psychosocial Goals
Related posts
Launch a Business for Under $500: Real Odds, 4 Founders
0dev.to AI
Google, Microsoft, OpenAI and Anthropic Back a Shared AI Training Opt-Out Signal
0dev.to AI
Did AI Decode the Voynich Manuscript? Claims Tested
0dev.to AI
AIGB 解决 Teletype.in 群组管理痛点
0dev.to AI

Qual a melhor arquitetura de dados para a sua aplicação de IA?
0dev.to AI
Connect a Stateless Streamable HTTP MCP Server to Claude, ChatGPT, and Gemini for Tour Route Research
0dev.to WebDev