Open Source
ToolBottleneckBench: Measuring Tool-Agent Performance under Controlled Pressures and Fixed Harness Budgets
arXiv cs.SE0 views
arXiv:2610.21098v1 Announce Type: cross Abstract: The success score of a tool agent reflects both its model and the budgets of the harness that runs it. We present ToolBottleneckBench, a benchmark that measures tool-agent performance under controlled pressures and fixed harness budgets. Synthetic enterprise tasks stay fixed while one pressure varies: the registered tool catalog, the visible tool menu, the certified workflow, or the encoding of returned records. An exact grader records the first unmet obligation of each failed episode and the harness limit, if any, that ended it. Across 21 model and harness configurations, the visible menu is the only one of the three pressures that act on the model with a large penalty, and this penalty tracks truncation at the completion cap. Raising the cap sixteenfold removes most of the penalty in the truncation-heavy configurations. Most failures end at a harness limit, and the failure record identifies the assigned pressure no more reliably than a count of tool calls. The repairs that help change which tools reach the model or who issues the calls, and only two repairs, neither improving model behavior, beat every configuration switch. We recommend reporting pressure effects with the harness budgets under which they were measured. We release the benchmark at https://github.com/zichuanxu/tool-bottleneck-bench.
Source:https://arxiv.org/abs/2610.21098PreviousWhere the SRAM Goes: Per-Symbol Attribution of Static Memory in FreeRTOS-C and Embassy-Rust IoT Firmware on a 64 KB RISC-V MicrocontrollerNextIndependent Data-Integrity Verification for LLM-Driven Functional Coverage Closure in Hardware Design
Related posts
GIMP 3.2.8 released with security fixes
0dev.to OpenSource

We got real Linux on iPhone!
0dev.to Linux
EveryOS - A desktop-style web GUI for Linux servers.
0dev.to Docker
Show HN: HN Atlas – Discover tools to better engage with Hacker News
0Hacker News Show
Show HN: TextoCAD – Text to editable CAD (STEP/STL), runs locally with your key
0Hacker News Show
Owen Sound: From Transit Budgets to Ride Targets
0dev.to JavaScript