AI & ML
Multimodal Data Comprehension: Understanding How Visual-Textual Chains of Information Influence Data Interpretation
arXiv cs.HC0 views
arXiv:2607.23489v2 Announce Type: replace Abstract: Visualizations and text often work together to support effective data communication. Despite this common paradigm, we know little about how the interplay of these modalities affects people's data comprehension. We present a novel experimental paradigm to investigate multimodal data comprehension---the process of people comprehending information from multimodal visual and textual data---across both crowdsourced and think-aloud environments. Our methodology employs two sequential chains for presenting multimodal information---a visualization-first chain and a text-first chain---asking people to describe the data presented iteratively. By comparing how people's data comprehension changes across the chain, we can assess the information contribution of each modality and how they shape subsequent comprehension. We found that the visualization-first chain facilitates exploratory comprehension with hypothesis-driven discovery, whereas the text-first chain yields confirmatory comprehension akin to framing effects where visualizations serve to reinforce and confirm observations drawn from text. Our findings provide empirical insights into multimodal information integration, with implications for designing more effective data-driven communication.
Source:https://arxiv.org/abs/2607.23489PreviousWhat to Distinguish and How? Opportunities and Challenges of Augmenting Multiple, Cluttered Objects in Complex Scenes for People with Low VisionNextNavSight in the Wild: Understanding Real-World Use of a Mobile Augmented Reality Application for People with Low Vision in Outdoor Navigation
Related posts
Introducing the Grid Protectors Program
0OpenAI Blog
The world has a new plan to save nature, but it still needs the money
0Ars Technica

Could data center backlash flip Florida?
0The Verge

Vivo’s first Pro Max phone proves its Ultra is still in a league of its own
0The Verge

Why Peacock is making a huge bet on YouTube
0The Verge
Ninja’s first-ever microwave just replaced two of my kitchen appliances at once
0TechCrunch