AI & ML
Function-Space Temporal Self-Distillation for Policy Optimization
arXiv stat.ML0 views
arXiv:2610.19687v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in language models, but scalar outcomes provide coarse supervision over response trajectories. Self-distillation offers denser token-level guidance by comparing an unassisted policy with a feedback-informed self-teacher, yet the teacher evolves with the student, creating unstable targets and motivating additional regularization. We introduce Function-Space Policy Optimization (FSPO), a single-policy framework that combines current privileged adaptation with function-space temporal consolidation. FSPO stores sparse historical next-token distributions as fixed temporal targets, providing temporal separation without maintaining a persistent teacher model. It further introduces privileged disagreement correction to explicitly suppress locally overvalued actions, reliability-adaptive grounding that strengthens reference supervision when self-generated evidence repeatedly fails, and quality and coverage-aware temporal memory that retains verified behaviors and improves privileged demonstrations. Across reasoning and tool-use benchmarks, FSPO achieves the highest average accuracy under a 5-hour training budget, improving over the strongest baseline by approximately 4% points on average while reducing peak memory from about 70 GB to 56 GB on Olmo3-7B-Instruct.
Source:https://arxiv.org/abs/2610.19687PreviousOn Global Joint Lower Bounds in Zeroth-Order Nonconvex OptimizationNextAnnealed L\'{e}vy Diffusion via Gaussian Surrogate Reverse Dynamics
Related posts

Affinity’s creative studio is now available on iPad for free
0The Verge

SpaceX could buy Boost Mobile as part of its big mobile carrier plans
0The Verge
Northwood Space raises $375M Series C at a $2B valuation from a16z and DFJ growth
0TechCrunch

The Download: the agentic web arrives and the drive for safer roads
0MIT Technology Review

Stop counting tokens, start counting what ships
0MIT Technology Review
OpenAI is finally merging ChatGPT’s chat and Work tabs
0TechCrunch