DesignLex
/人机交互/multimodal interaction
论文进阶

multimodal interaction

多模态交互:结合多种输入与输出通道(如语音、注视、手势、触摸、视觉叠加等)的交互方式,让人与计算系统的沟通更自然、灵活。

English Definition

"Interaction that combines multiple input and/or output modalities (e.g., speech, gaze, gesture, touch, visual overlays), enabling more natural and flexible communication with computational systems."

用法说明 · 针对中文母语者

在 HCI 中是相对于单模态(如纯文本或纯 GUI)的设计范式,不要与机器学习中的 multimodal learning 或 multimodal model 简单等同。

真实用例 · 3

"Supporting humans and AI in fluidly establishing, shifting, and maintaining shared attention through multimodal signals such as gaze, gestures, and AR highlights."

论文Pipeline extracted — review needed·Joint Attention Expression 组件的描述

"We instantiate this framework in an AR-based prototype equipped with a real-time multimodal pipeline."

论文Pipeline extracted — review needed·Contribution (2) 中描述系统实现

"Multimodal expressions, including natural language, gaze, and gestures, offered unique contributions to the establishment of common ground."

论文Pipeline extracted — review needed·用户研究结果总结
由 pipeline 自动采集,待人工 review