Open Source
Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations
arXiv eess.SP0 views
arXiv:2608.07385v2 Announce Type: replace-cross Abstract: Learning disentangled representations is key for general-purpose multi-modal wearable models, yet existing approaches rarely combine classification, disentanglement, fusion, and generation of heterogeneous multi-modal time series in a single model. We introduce Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs), a framework that learns multi-purpose representations from arbitrarily many modalities with a single shared encoder-decoder. OmniDecVAEs extend DecVAEs by learning time-frequency latent subspaces per modality through a multi-view self-supervised decomposition loss and a component-wise conditional decoder. On the omni-modal human activity recognition (HAR) HARWE dataset (up to thirty channels) and the public UCI HAR and WESAD datasets, OmniDecVAEs learn the most disentangled representations for unseen subjects (DCI disentanglement up to 3.0 percentage points (pp) higher). On HARWE, they improve subject-dependent HAR and identity recognition by 1.1 pp and 5.8 pp, and outperform supervised end-to-end models by +31.6 pp and +22.3 pp on unseen subjects of multi-device datasets. Moreover, their generative capacity improves reconstruction mean squared error over multi-modal VAEs (34-74% improvement) and their synthetic data improve affect recognition on unseen subjects (2 pp). With 4.1M parameters, invariant to the number of modalities (21xfewer than a multi-branch VAE), OmniDecVAEs are scalable, single-model full-stack learners for wearable representations.
Source:https://arxiv.org/abs/2608.07385PreviousSpectral Vision Transformer for Efficient Tokenization with Limited DataNextBenchmarking Open-Source O-RAN and 5G NTN Stacks on Embedded ARM Platforms for Space Payloads
Related posts
Are You Buying a Company or a model_id?
0dev.to Discuss
How to Test a Resistor, Diode, LED and Capacitor with a Multimeter (Before You Blame Your Code)
0dev.to Beginners
Spectral Vision Transformer for Efficient Tokenization with Limited Data
0arXiv eess.SP
Gaussian Quasi-Likelihood Estimation for Nonsynchronously Observed L\'evy processes
0arXiv stat.TH
Triple Overlap in Pipelined Krylov Methods on Multi-GPU Systems
0arXiv cs.DC
Rhythm-TTS: High-Fidelity Shortcut Latent Modeling and Probabilistic Rhythm Generation for Zero-Shot Text-to-Speech
0arXiv cs.SD