출처: arXiv · cs.CV원문 보기 ↗
원문 저작권은 출처에 있습니다. 이 사이트는 수집, 번역 또는 형식 정리만 합니다.
사실 흐름
arXiv:2608.11474v1 Announce Type: new Abstract: Object Hallucination in large vision-language models (LVLMs), where models generate non-factual content about input images, remains a critical barrier to their reliability
해설과 영향
测试时幻觉控制:大视觉语言模型可靠性的新研究方向
从技术背景来看,物体幻觉并非新问题。此前已有大量研究试图通过改进训练数据、引入外部知识或调整解码策略来缓解该现象。而“测试时控制”这一思路的侧重点在于:不重新训练模型,而是在推理阶段对生成过程施加干预或约束,以降低幻觉输出概率。该论文的具体方法、实验设置与效果数据在摘要中均未展开,[原文未提供]更多细节。
值得注意的是,该论文发布同日,arXiv 上还出现了另一篇关于多 LLM 智能体系统的研究(arXiv:2608.11207),其关注点在于多个目标结构对立的 LLM 智能体在多轮交互中缺乏共享目标函数所引发的问题。两者虽同属大模型可靠性范畴,但研究对象与路径不同:前者针对视觉-语言跨模态生成中的事实一致性,后者聚焦多智能体对话协作的动态治理。
对于关注 LVLMs 落地应用的读者而言,测试时幻觉控制的意义在于:它提供了一种无需高昂再训练成本即可提升模型可信度的潜在路径。若该方法被验证有效,将有助于视觉问答、图像描述、文档理解等场景中减少模型“看图说话”时的虚构内容。不过,由于摘要未披露实验数据与对比基线,其实际效果尚待论文全文公开后进一步评估。
참고 자료
출처 원문
arXiv:2608.11474v1 Announce Type: new Abstract: Object Hallucination in large vision-language models (LVLMs), where models generate non-factual content about input images, remains a critical barrier to their reliability in real-world applications. Existing mitigation strategies can be categorized into training-based and training-free methods. Training-based methods often achieve strong performance but are costly, requiring extensive computational resources, large-scale data, and time-consuming fine-tuning. Training-free approaches are particularly appealing due to their efficiency. However, existing training-free methods either require multiple decoding rounds, which adds computational overhead, or modify internal states in a model-specific way that risks degrading pretrained knowledge. We propose Test-Time Hallucination Mitigation (TTH) method, a novel training-free method that addresses both limitations. TTH introduces a token-validator module, implemented as a zero-shot Multi-Modal Classifier (MMC), to generate auxiliary logits grounded in the input image. These logits are fused with the original LVLM outputs at the token level for object tokens selected from a candidate pool. An entropy-based weighting scheme is then applied to enable robust and accurate predictions. Extensive experiments across multiple LVLM families and diverse benchmarks demonstrate that TTH consistently improves accuracy and robustness, underscoring its generalizability and practical effectiveness. Code is released at https://github.com/Mehran-TAM/TTH