AI & ML
From Bytes to Behavior: Request-Level Attribution from Live GPU Hidden States
arXiv cs.CR0 views
arXiv:2611.18272v1 Announce Type: new Abstract: Existing memory forensics for machine learning systems typically targets static artifacts, such as model weights or prompt logs, rather than the transient computational states of a live inference process. This gap limits an examiner's ability to analyze the specific behavior of a running large language model (LLM) service that cannot be restarted or modified. We present a method for acquiring per-layer residual-stream activations from a live GPU process using a non-modifying debugger, without requiring prior knowledge of the model's architecture parameters or weights. The approach employs a general six-stage process with platform-specific adapters, utilizing either a PyTorch allocator clock or a kernel-launch clock to identify and read tensor allocations from video memory. We validate this acquisition on 33 open-source decoder-only models across nine architecture classes, demonstrating that the recovered bytes are byte-exact and usable for downstream analysis. As a use case, we apply linear probes trained on standard framework activations to the recovered states, successfully distinguishing refusal from compliance with an AUROC of 0.88 on held-out prompts. This work establishes that live, observational access to intermediate hidden states is feasible on named inference platforms, providing a new source of evidence for examining the operational behavior of deployed AI systems.
Source:https://arxiv.org/abs/2611.18272PreviousForget the Lineage: Forward-Secure, Lineage-Complete Cryptographic Erasure for LLM-Agent MemoryNextPrivacyPR: A Dataset of Real-World Privacy Fixes for Benchmarking Privacy-Aware Code Review
Related posts
How to Trim Audio for an Online Course Lesson — Free, in Your Browser
0dev.to WebDev
Chemical Anchor Fasteners: A Practical Guide to Stainless Steel Chemical Anchor Studs
0dev.to WebDev
Automating Notion Desktop App Installation on Arch Linux
0dev.to Linux
existence-trace: a journal your AI reads, and has to prove it used
0dev.to Python
A 30-question regression suite for your customer-service chatbot (with a tiny Python runner)
0dev.to Python
The boring-infrastructure approach to running AI agents: cron, env files, launchd, and a markdown decision log
0dev.to DevOps