Backend
OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation
arXiv cs.CR0 views
arXiv:2610.31616v1 Announce Type: cross Abstract: On-policy distillation has been widely adopted for alleviating the distribution mismatch of off-policy distillation. Self-distillation further removes the need for an external teacher by using the student's own outputs, enhanced with extra context or demonstrations, as supervision. This teacher-free design makes on-policy self-distillation practical, yet its privacy risks remain largely unexplored. In particular, using two coupled views of the same training example may strengthen its memorization. Motivated by this, we present the first study of membership inference attacks against on-policy self-distillation. We investigate both gray-box and black-box threat models and propose a unified attack framework, named OP-Print. OP-Print builds on our finding that the supervision imprint in on-policy self-distillation is sparse and structured, concentrating on uncertainty-sensitive correction sites rather than across the entire rollout. Exploiting this property, we construct attack signals tailored to the two access levels: in the gray-box setting, OP-Print directly aggregates the privileged teacher-student residual correction along the same rollout; in the black-box setting, it measures restart-based output-basin stability as an observable surrogate for the hidden correction process. Across diverse models and distillation schemes, all baseline attacks remain essentially at random guessing even under gray-box access, whereas OP-Print is broadly validated to be effective in both gray-box and black-box settings.
Source:https://arxiv.org/abs/2610.31616PreviousEfficient and Optimal Learning of Discrete Distributions with Person-Level PrivacyNextHow to Backdoor Image Knowledge Distillation
Related posts
Faking a shortwave radio in the browser: static, heterodyne whistles and stations that bleed into each other (Web Audio)
0dev.to

Beyond Turing: The Rise of Neuromorphic Computing and Spiking Neural Networks
0dev.to
To plan or not to plan: the Terraform file that leaked every password we had
0dev.to

Overwrote a secret by mistake? Writing, pulling and rolling back keys
0dev.to
Beyond Traditional Automation: How Intelligent Industrial Systems Are Transforming Operational Decision-Making
0dev.to Data
Chrome Is Open Source and Still Hard to Reverse Engineer. Here's Where the Real Work Is!
0dev.to Discuss