來源:arXiv · cs.AI查看原文 ↗
原文著作權歸來源方所有,本站僅作收錄、翻譯或格式整理。
事實脈絡
arXiv:2608.11216v1 Announce Type: new Abstract: World modeling is an unsettled field: architectures, training objectives, and state representations interact in complex ways, and no single recipe dominates across environ
解讀與影響
AutoWorldModel-Bench:面向自动化世界模型研究的状态中心基准
该基准的核心思路是将评估焦点从最终任务表现转向世界模型对"状态"本身的建模质量。传统世界模型评测往往以智能体在特定任务中的得分作为间接指标,但这种方法难以区分性能提升究竟来自更准确的世界模型,还是来自下游策略的改进。AutoWorldModel-Bench 通过直接考察模型对状态转移的预测能力、状态表征的紧凑性与可复用性等维度,试图将世界模型的质量评估从任务耦合中解耦出来。摘要明确指出,当前领域内"架构、训练目标和状态表征以复杂方式相互作用",而该基准的目标正是为这种交互提供可量化的比较基础。
从同日发布的相邻研究来看,世界模型与智能体系统的评估正呈现出从"单一任务得分"向"结构性与因果性指标"迁移的趋势。例如,Detecting a Route Flip Is Easier Than Knowing Whether to Fix It 关注量化扰动下 MoE 路由的因果性损伤检测,Dynamic Governance of Multi-LLM Agent Systems 则探讨多智能体系统中缺乏共享目标函数时的协作治理问题。这些工作虽主题各异,但共同反映出研究者对"系统内部机制可观测性"的日益重视——这与 AutoWorldModel-Bench 强调状态表征质量而非终端得分的取向一脉相承。
值得注意的是,AutoWorldModel-Bench 的"自动化"定位暗示其评估流程可能减少人工设计干预。不过,摘要中关于基准的具体构成、覆盖的环境类型以及自动化评估的实现细节,原文未提供更多信息。作为一篇新发布的 arXiv 预印本,该基准的实际可用性与社区接受度仍有待后续版本的补充和同行的独立验证。
參考來源
來源原文
arXiv:2608.11216v1 Announce Type: new Abstract: World modeling is an unsettled field: architectures, training objectives, and state representations interact in complex ways, and no single recipe dominates across environments. This makes it an ideal testbed for AI coding agents acting as autonomous researchers--a setting in which the improvement direction is not specified in advance, unlike the engineering-to-spec tasks that dominate current agent benchmarks. We introduce AutoWorldModel-Bench, a closed-loop benchmark in which frontier coding agents autonomously improve a provided world-model starter under a fixed compute budget. The benchmark spans eight game environments under a unified structured-state representation--ground-truth entity state extracted from each game and consumed through a shared tensor format--which isolates dynamics modeling from perception and enables minutes-per-run iteration. Across 64 sessions, Codex-5.4 and Claude Opus 4.6 improve their starter on 63; in 91% of sessions the winning edit is a non-trivial research-style modification--a new objective, representation, rollout procedure, or architectural change--rather than a hyperparameter tweak. Our benchmark offers a setting in which frontier coding agents can be evaluated on open-ended research rather than engineering-to-spec problems.