AI & ML
Auditing LLM Pseudo-Label Distillation for Review Informativeness at E-Commerce Scale
arXiv cs.IR0 views
arXiv:2611.13797v1 Announce Type: new Abstract: We study hard-label distillation from Gemini 3.1 Flash-Lite to compact encoders on 5.34 million Turkish marketplace reviews. A strict new-SKU temporal test measures teacher fidelity; a separate, teacher-label-blind audit by one company annotator measures editorial agreement. An H3 category prefix slightly improves teacher fidelity on the full new-SKU test. On the 16,000-review chronological audit subset, it instead reduces agreement with both Gemini and the annotator, with a clearer paired difference for CORN than for cross entropy. Thus the context effect depends on the evaluation population, and teacher fidelity alone does not establish editorial alignment. Under the measured offline configurations, a single-L4 student pipeline processes reviews 50.2 times faster than the teacher pipeline.
Source：https://arxiv.org/abs/2611.13797PreviousExploiting Feature-invariant Scaling Laws for Efficient A/B Experimentation in Recommendation SystemsNextControlled Knowledge Engineering for Taxonomy Induction with Tool-Grounded LLM Agents and Runtime-Adaptive Policies
Related posts
I wanted one screen that said whether a kids' app was worth installing
0dev.to
Before I Could Build GPT, I Had to Teach a Machine to Learn
0dev.to
Cogent and OpenAI build cyber defense factories for enterprises
0OpenAI Blog
ChatGPT Plus for veterans and servicemembers
0OpenAI Blog
Hitting 200 Strokes: What the Signature Maker Does at Its Stroke Limit
0dev.to
A Lightweight, Open-Source Network Digital Twin Blueprint for IoT-Edge Environments
0arXiv cs.NI