Backend
TailGuard-GDS: Tail-Safe Control of GPU Storage Data Movement
arXiv cs.DC0 views
arXiv:2611.17824v1 Announce Type: new Abstract: AI runtimes move state between NVMe storage and GPU memory on critical paths. GPUDirect Storage (GDS) removes host staging, yet our eight-H100 measurements show it is not consistently tail-optimal for semantically identical GPU-buffer operations. The preferred path varies with operation, request size, concurrency, placement, and resource contention; bandwidth and tail-latency winners can disagree. Frameworks specify what moves and when, while GPU I/O stacks expose mechanisms; neither combines phase semantics with measured tail behavior to choose a delivery plan. We present TailGuard-GDS, a runtime that selects a semantically equivalent plan per application phase. It profiles plans varying backend, request shape, and placement, minimizes GPU-visible tail latency under phase constraints, and revisits decisions as conditions change. Resource reuse keeps policy lookup and switching at microsecond-scale overhead. Integrated with PyTorch FSDP and DeepSpeed ZeRO-3, TailGuard reduces FSDP step p95 by 18.4% and GPU-idle p95 by 25.3% versus the strongest fixed path. Across five eight-GPU ZeRO-3 workloads, step p95 falls 7.9-24.6% versus fixed host staging and 28.4-42.6% versus fixed GDS. Against tuned DeepSpeed AIO, TailGuard reduces step p95 by 5.6% and raises estimated throughput by 6.1%.
Source:https://arxiv.org/abs/2611.17824PreviousSuperfluid: Local AI Server for Distributed LLM InferenceNextThe Exact Stabilization Time of Self-Stabilizing Coloring: Mean, Variance, and Distribution
Related posts
How to redesign a website without losing Google rankings
0dev.to UX
What a Good Sandboarding Booking Website Needs in Dubai
0dev.to UX
Why GitHub Actions warns about Node 20 when your project uses Node 24
0dev.to GitHub
165 "completed" tasks. 21 left a clickable link. 11 of those links are already dead.
0dev.to GitHub
How to Push to GitHub Without Git: The API Workflow Behind My Solo Shipping Habit
0dev.to GitHub
How to get YouTube channel transcripts in bulk
0dev.to Data