Nvidia used Factory’s Droid coding-agent harness to generate roughly 20,000 rows of post-training data for Nemotron 3 Ultra, the 550-billion-parameter open model it released in June.
Capturing reproducible agent failures
In a Wednesday case study, Factory detailed how Nvidia deployed Droid inside an air-gapped network and instrumented the harness to capture reproducible failures in instruction following, tool calling, and error recovery for model post-training.
Droid relies on Nemotron to maintain a structured task plan throughout execution. Factory found problems with tool calls and argument formatting, as well as instances where the model continued working without recovering from errors.
Factory also evaluated Nemotron against proprietary datasets, including Legacy Bench, to isolate individual decisions behind unsuccessful task execution.
“Factory collaborated with us to analyze those traces to improve the model’s planning, tool use, and error recovery.”
“Factory collaborated with us to analyze those traces to improve the model’s planning, tool use, and error recovery,” Jian Zhang, Nvidia’s director of applied research for Nemotron post-training, said in the case study.
What PivotRL changes
Nvidia turned to PivotRL, a reinforcement learning method introduced in March that trains models on critical decisions within existing supervised fine-tuning trajectories without the expense of full multi-turn rollouts. Its reward mechanism evaluates whether an action achieves the intended outcome rather than whether it matches the reference example.
On agentic coding tasks, PivotRL achieved accuracy comparable to end-to-end reinforcement learning with one-quarter as many rollout turns.
On agentic coding tasks, PivotRL achieved accuracy comparable to end-to-end reinforcement learning with one-quarter as many rollout turns.
PivotRL outperformed standard supervised fine-tuning on identical datasets by an average of 4.17% across four agentic domains and 10.04% on out-of-domain non-agentic tasks. Nvidia previously incorporated PivotRL into production-scale post-training for Nemotron 3 Super. For Ultra, the company returned updated checkpoints to Factory for further evaluation.
Guarding against harness overfitting
Nvidia varied Droid’s configuration and incorporated data from several agent environments to reduce the risk of harness-specific overfitting.
The company’s NeMo RL documentation details a multistage pipeline that combines reinforcement learning with verifiable rewards, human feedback and multi-teacher on-policy distillation. Terminal-use training configurations include OpenCode, Codex and an environment identified as agent006 alongside Droid-specific PivotRL data.
The distillation process transfers capabilities from teacher models trained across different technical domains into the student model. Nvidia has published its reinforcement learning blends along with instructions for reproducing the pipeline on GB200 NVL72 hardware.
Nemotron 3 Ultra’s Terminal-Bench 2.0 score increased from 38.5% to 49%…
According to Factory, Nemotron 3 Ultra’s Terminal-Bench 2.0 score increased from 38.5% to 49%, while a preliminary checkpoint of Nemotron 3.5 Super VL improved from 52.11% to 60.4% on Terminal-Bench 2.1.
Factory has not published an ablation study isolating its contribution to either benchmark gain or disclosed before-and-after error rates for individual tool-use failures.
Harness-dependent performance gains also surfaced in evaluations of Google’s Gemini 4 Neon, which performed substantially better inside Google’s Antigravity harness.
Training harness-specific adapters
Hugging Face’s Harness Hub team took a different approach in a November 6 technical post. The team used reinforcement learning to train separate LoRA adapters for Pi, OpenCode and Hermes without modifying the harnesses or base model weights.
On Tuesday, Hugging Face CEO Clem Delangue posted on X that a version of Qwen 3.8 27B optimized for Pi improved its Terminal-Bench 2.1 score from 39.5% to 55.8% and reduced tool calls per task by 55% to 62%.
The team also developed safeagents, a rule-based proxy that inspects tool calls to prevent reward hacking during training.
Neither Nvidia nor Hugging Face has published comparable evaluations that quantify the performance and computational trade-offs between harness-specific adaptation and multi-harness post-training.
The post Nvidia used 20,000 training examples to fix its AI’s mistakes. Its coding score jumped. appeared first on The New Stack.