AI & ML
Fast and Actionable Classification of HPC I/O Time Series
arXiv cs.DC0 views
arXiv:2611.09646v1 Announce Type: new Abstract: Input/output (I/O) is a major bottleneck for highperformance computing (HPC) performance. One of the challenges is the contention between jobs when sharing the parallel file system. However, tackling this issue depends on knowing the temporal behavior of jobs. In this paper, we propose a clusteringbased approach to characterize jobs' temporal I/O behavior. We propose SAGE, a lightweight method that cuts I/O traces into short windows and learns an alphabet from them. The trace then becomes a sequence of symbols, which we call ''genome'', and we use clustering to separate them into classes. We evaluate SAGE and its two-stage variant SAGE-2 on 819,771 real traces from 2 HPC systems. We compare them against other techniques from the state of the art on three metrics, including one we propose. Compared to K-shape, the best baseline available, SAGE-2 lowers the median intra-cluster metrics by 9.1% on average, and achieves this in 2 minutes of clustering time versus 3.8 hours, a 109x speedup. More importantly, clusters created by SAGE and SAGE-2 carry enough information to act on: we evaluate the use of the cluster centroids for I/O scheduling, and show that our approach can decrease I/O stretch by up to 24% compared to the state of the art (round-robin placement).
Source:https://arxiv.org/abs/2611.09646PreviousTowards Sustainable Federated Learning via Carbon-Aware Client SelectionNextConcurrent Use of FP32 and FP64 Computing Units on GPUs and Mixed-Precision Placement in WENO Reconstruction
Related posts
A planner that cannot invent a capability
0dev.to OpenSource
A small business running 8 projects on Claude Code for 30 days: rules ignored, projects crossing into each other, and what finally helped
0dev.to AI
L'IA trouve les peptides, la VR les révèle : pourquoi Antigravity change la donne pour la découverte de médicaments
0dev.to AI
Why Tesseract Fails on Cursive Handwriting (And How We Built a Two-Tier Vision Pipeline in Node.js)
0dev.to AI
OFX looks like XML. It isn't, and QuickBooks will reject it
0dev.to WebDev
Before You Buy GEO Monitoring Software, Build a 50-Line Baseline
0dev.to Python