Backend
Prioritizing Microservice Requests via Retries
arXiv cs.DC0 views
arXiv:2610.20567v1 Announce Type: cross Abstract: Cloud services rapidly shift towards independently-managed microservices architectures. This independence may lead to temporary microservices miscoordination, resulting in overloads and rejections of valuable transactions. This weakness may further be exploited by attackers to launch overload attacks inflicting performance and cost damages. Retry operations are a key machinery aiming at overcoming seldom rejections. However, during overload periods they: a) Fail protecting valuable transactions from rejections, and b) Trigger large-scale performance-degrading retry storms. We propose simple modifications in retry operations, overcoming these deficiencies by transforming retries into a rejection prioritization mechanism, DiffTry (Differential Retries). It is based on a simple, lightweight, distributed and stateless design and requires neither overload sensing nor microservices coordination. We develop an analytical model capturing DiffTry's effectiveness for two services in tandem, then generalize it to networks of services. Our theoretical framework lets practitioners quantify, for a given network, the resulting prioritization and rejection rates, and yields formal guarantees that DiffTry substantially reduces the impact of both incidental and adversarial orchestrated overloads. We validate these predictions in simulation and on a live Kubernetes deployment, where DiffTry is a simple drop-in service-mesh plugin requiring no application changes; the measured rejection closely matches the analysis.
Source:https://arxiv.org/abs/2610.20567PreviousDelta Tokenizer: Reference-Identical Tokenization of Session Deltas and Segment-Parallelism for Long Prompts on Multicore CPUsNextKonflux: Optimized Function Fusion for Serverless Applications
Related posts
I used to dream about coding on the train. Now I'm managing agents from my phone. 🚉🍁🍂
0dev.to Productivity
Streamlining GitHub Activity: Why Lost Filters Impact Software Engineering Measurement
0dev.to Productivity

The Roofline Model for LLM Inference
0dev.to Machine Learning
Deduplicate Your RAG Corpus Without Losing Its Citations
0dev.to Machine Learning

scm2cpp-learn: 44 scikit-learn estimators, same results, up to ~1000x faster
0dev.to Machine Learning
Marvell is buying Lumentum for $115B: what the optical bet means for AI cluster power, and what a fixed 4.5x exchange ratio really locks in
0dev.to Machine Learning