ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual
  Perception in VLMs

Xiyao Wang; Zhengyuan Yang; Chao Feng; Yongyuan Liang; Yuhang Zhou; Xiaoyu Liu; Ziyi Zang; Ming Li; Chung-Ching Lin; Kevin Lin; Linjie Li; Furong Huang; Lijuan Wang

Paper · arXiv 2506.10128

ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs

Xiyao Wang, Zhengyuan Yang, Chao Feng, Yongyuan Liang, Yuhang Zhou, Xiaoyu Liu, Ziyi Zang, Ming Li, Chung-Ching Lin, Kevin Lin, Linjie Li, Furong Huang, Lijuan Wang

22 upvotesJune 11, 2025arXiv 预印本

AI 摘要

ViCrit, an RL task for fine-tuning VLMs, improves visual perception by training models to detect subtle hallucinations in image captions, with gains transferable to various visual domains.

reinforcement learning (RL)large language models (LLMs)vision-language models (VLMs)visual caption hallucination criticperceptual difficultybinary rewardexact-match rewardViCrit-Benchabstract image reasoningvisual math

Abstract

Reinforcement learning (RL) has shown great effectiveness for fine-tuning large language models (LLMs) using tasks that are challenging yet easily verifiable, such as math reasoning or code generation. However, extending this success to visual perception in vision-language models (VLMs) has been impeded by the scarcity of vision-centric tasks that are simultaneously challenging and unambiguously verifiable. To this end, we introduce ViCrit (Visual Caption Hallucination Critic), an RL proxy task that trains VLMs to localize a subtle, synthetic visual hallucination injected into paragraphs of human-written image captions. Starting from a 200-word captions, we inject a single, subtle visual description error-altering a few words on objects, attributes, counts, or spatial relations-and task the model to pinpoint the corrupted span given the image and the modified caption. This formulation preserves the full perceptual difficulty while providing a binary, exact-match reward that is easy to compute and unambiguous. Models trained with the ViCrit Task exhibit substantial gains across a variety of VL benchmarks. Crucially, the improvements transfer beyond natural-image training data to abstract image reasoning and visual math, showing promises of learning to perceive rather than barely memorizing seen objects. To facilitate evaluation, we further introduce ViCrit-Bench, a category-balanced diagnostic benchmark that systematically probes perception errors across diverse image domains and error types. Together, our results demonstrate that fine-grained hallucination criticism is an effective and generalizable objective for enhancing visual perception in VLMs.