DesignLex
/人机交互/Computer-Using Agent (CUA)
论文专业

Computer-Using Agent (CUA)

计算机使用代理(Computer-Using Agent, CUA),基于大型多模态模型构建的自主 AI 代理,能像人一样直接操作图形界面(点击按钮、打开窗口、执行界面级动作),代替用户完成电脑操作任务。

English Definition

"An autonomous AI agent, typically built on large multimodal models, that directly manipulates graphical user interfaces (e.g., opening windows, clicking buttons, executing interface-level actions) to perform tasks on behalf of users, operating computers in a human-like manner."

用法说明 · 针对中文母语者

CUA 是 2024-2025 年随着多模态大模型兴起而出现的新概念,强调 AI 不只是回答问题,而是能'使用'电脑。OpenAI 的 Operator(2025 年 1 月发布)是首个被广泛引用的 CUA 实现。

真实用例 · 3

"The recent advent of Computer-Using Agents (CUA) based on Large Multimodal Models, which can directly manipulate graphical user interfaces, offers new opportunities for such accessibility."

论文Pipeline extracted — review needed·在引言中介绍 CUA 作为解决视障用户在线购物障碍的新机会

"Recent advances in Large Multimodal Model (LMM) have led to the emergence of Computer-Using Agent (CUA), which can operate computers like human users (e.g., opening windows, clicking buttons, and executing interface-level actions)."

论文Pipeline extracted — review needed·解释 CUA 与传统 GUI 自动化的区别

"We employed Operator, a CUA model released by OpenAI [75] in January 2025, and implemented a Text-to-Speech (TTS) module to automatically detect and vocalize the model's responses."

论文Pipeline extracted — review needed·方法部分说明实验所用的具体 CUA 模型
由 pipeline 自动采集,待人工 review