Backend
On the Optimality of Implicit Feedback in Markov Decision Processes over Imperfect Links
arXiv cs.SY0 views
arXiv:2611.06365v1 Announce Type: new Abstract: Goal-oriented communication protocols are explicitly designed to support the objectives of target applications, such as the monitoring and control of remote processes, while minimizing data traffic. This study considers a transmitter node that observes the environment state and sends it to a remote agent, which then performs control actions. This problem can be modeled as a remote Partially Observable Markov Decision Process (POMDP), a subclass of decentralized POMDPs. Existing studies limit transmissions through communication costs, but neglect the effect of channel impairments and non-ideal or delayed feedback. In this work, we study the optimal joint transmission and control policies over lossy forward channels without explicit acknowledgments. We introduce \emph{revealing} policies, under which the receiver's action, observed directly by the transmitter, discloses the outcome of the forward transmission, keeping the two agents synchronized without a feedback channel. We derive sufficient conditions for the optimality of time-invariant policies in this implicit feedback regime for general POMDPs, along with strictly weaker sufficient conditions in the special case of contextual bandits. Our simulations over a family of remote POMDPs show that these conditions are conservative, and revealing policies are optimal over a substantially wider parameter range than the conditions warrant.
Source:https://arxiv.org/abs/2611.06365PreviousTopology Identification of Directed Signed Networks via Repelling LaplaciansNextSpatial Differences Analysis of Nodal Frequency Considering Frequency-Voltage Coupling
Related posts
Cloud Storage CORS Support for Direct Browser Receipt Uploads (A Go Retention Boundary)
0dev.to Go
5 Ways to Publish Timed Caption Segments with Reconnect Handling in Live Rooms
0dev.to Go

Efficiently Growing Unowned Maps in Go: Avoiding Resizes and Reducing Memory Overhead
0dev.to Go
Syncing 1C and Bitrix24 Without Duplicates: Designing an Idempotent Exchange
0dev.to Database
Kimball's Data Warehouse Architecture: Evaluating the Challenges of Conformed Data against the Inmon Model
0arXiv cs.DB
COL-Trees: Efficient Hierarchical Object Search in Spatial Networks
0arXiv cs.DB