Through the Looking Glass: Common Sense Consistency Evaluation of Weird
  Images

Elisei Rykov; Kseniia Petrushina; Kseniia Titova; Anton Razzhigaev; Alexander Panchenko; Vasily Konovalov

返回文献探索

Paper · arXiv 2505.07704

Through the Looking Glass: Common Sense Consistency Evaluation of Weird Images

Elisei Rykov, Kseniia Petrushina, Kseniia Titova, Anton Razzhigaev, Alexander Panchenko, Vasily Konovalov

29 upvotesMay 12, 2025arXiv 预印本

AI 摘要

A new method, Through the Looking Glass (TLG), uses Large Vision-Language Models and Transformer-based encoders to improve image common sense consistency assessment on the WHOOPS! and WEIRD datasets.

Large Vision-Language Models (LVLMs)Transformer-based encoderattention-pooling classifierWHOOPS!WEIRD datasets

Abstract

Measuring how real images look is a complex task in artificial intelligence research. For example, an image of a boy with a vacuum cleaner in a desert violates common sense. We introduce a novel method, which we call Through the Looking Glass (TLG), to assess image common sense consistency using Large Vision-Language Models (LVLMs) and Transformer-based encoder. By leveraging LVLMs to extract atomic facts from these images, we obtain a mix of accurate facts. We proceed by fine-tuning a compact attention-pooling classifier over encoded atomic facts. Our TLG has achieved a new state-of-the-art performance on the WHOOPS! and WEIRD datasets while leveraging a compact fine-tuning component.