Backend
Characterizing Client-Independent Stationary Bias for SCAFFOLD and FedAvg in Federated Learning
arXiv stat.TH0 views
arXiv:2608.26765v2 Announce Type: replace-cross Abstract: Even when all clients optimize the same strongly convex objective, the global model parameters produced by constant-step federated learning may still have a stationary mean that differs from the minimizer. Does increasing the number of clients eliminate this bias, or can some components persist? Answering this question clarifies the limits of client averaging. To this end, we study two federated learning algorithms, \Scaf{} and matched \FedAvg{}, under full participation, a sufficiently smooth objective, and independent bounded additive gradient noise. Analyzing global stationary balance and within-round local fluctuations yields multidimensional expansions with a shared $\gamma/N$ term and explicit client-independent $\gamma^2$ coefficients, where $\gamma$ is the step size and $N$ is the client count. The \Scaf{} coefficient combines the ordinary local-update contribution found in \FedAvg{} with an additional persistent-control contribution. Although zero-sum controls cancel from the direct server increment, they change local second moments, which interact with third-order curvature to shift the stationary mean. For fixed dimension $d$ and local-step count $H$, a remainder of $O_{H,d}(\gamma^2/N+\gamma^3)$, uniform in $N$ for sufficiently small $\gamma$, identifies the second-order coefficients along arbitrary joint small-step, large-client limits after removing the shared leading term. Controlled experiments assess the predicted coefficients. In real-data analyses, finite-tail mean displacement can account for a substantial share of averaged-model parameter error, and a post-hoc comparison shows directional agreement with the derived curvature--noise structure.
Source:https://arxiv.org/abs/2608.26765PreviousOn the Role of Normalization in Binary Iterative Hard Thresholding for 1-bit Compressed SensingNextMutual Information Constrained Chernoff Bottleneck
Related posts
When a document assistant refuses: quota, context, and the missing model call
0dev.to C#
Humanize Perplexity Agent API Output in C#: A .NET 10 Linter for Orphan Citations and Source Talk + 5 Free Tools (2026)
0dev.to C#
Treat Your Payment Details Like a Public API: An IBAN in Your Own Name for Developers Paid From Abroad
0dev.to Career
Can You Use AI in Coding Interviews? What Changed in 2026
0dev.to Career
Legacy PHP Application Modernization Cost: What to Expect
0dev.to PHP
How Will Junior Developers Gain Experience in the AI Age?
0dev.to Beginners