AI & ML
Characterizing Robustness in Semantic Code Clone Detection
arXiv cs.SE0 views
arXiv:2606.25272v2 Announce Type: replace Abstract: Semantic code clone detection seeks to recognize shared functionality across different implementations, making robustness to behavior-preserving changes central to its effectiveness. Prior studies have exposed failures under such changes, motivating a systematic examination of how sensitivity varies with the rewrites used for evaluation. We present Systematic Code Transformation (SCT), a framework organizing 51 operators by code level, target, and edit category, and evaluate 14 token, tree, graph, and prompt-based detectors on BigCloneBench, Google Code Jam, and OJClone. By comparing transformation groups, repeated application, and alternative constructions, we connect overall performance changes with specific rewriting conditions. Of 25 token, tree, and graph detector and dataset combinations starting above 90% F1, 21 lose at least 10 percentage points under the composition of all operators. These losses vary with transformation type and application intensity, but are not determined by edit counts or code growth alone. Even at matched locations and edit counts, alternative constructions can expose different errors despite similar aggregate losses. Further analysis distinguishes reduced score separability from changed fixed-threshold decisions with retained ranking ability, revealing different scope for threshold recovery. Overall, these findings show why benchmark effectiveness needs to extend to stable discrimination across equivalent implementations and how systematic transformation testing can assess that capability.
Source:https://arxiv.org/abs/2606.25272PreviousAI Slop and the Software CommonsNextLarge Language Models for Multi-Lingual Equivalent Mutant Detection: An Extended Empirical Study
Related posts

The Download: a humanoid cleans house and Samsung builds for the AI boom
0MIT Technology Review

How Remedy designed Control Resonant’s wildest mission: ‘Anything kind of goes’
0The Verge

Four prompts that make AI images look like real photos (tested on Nano Banana and GPT Image)
0dev.to

OLED iPad Mini, M6 iMac, and New MacBook Pros Available From Today
0MacRumors

Apple Music Classical Updated With New Icon and Refreshed Design
0MacRumors
Tesla to wind down Cybertruck production as Elon Musk focuses on robotics and AI
0Hacker News New