Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding
Saved in:
| Published in: | arXiv.org (Nov 28, 2023), p. n/a |
|---|---|
| Main Author: | |
| Other Authors: | , , , , , |
| Published: |
Cornell University Library, arXiv.org
|
| Subjects: | |
| Online Access: | Citation/Abstract Full text outside of ProQuest |
| Tags: |
No Tags, Be the first to tag this record!
|
MARC
| LEADER | 00000nab a2200000uu 4500 | ||
|---|---|---|---|
| 001 | 2895042811 | ||
| 003 | UK-CbPIL | ||
| 022 | |a 2331-8422 | ||
| 035 | |a 2895042811 | ||
| 045 | 0 | |b d20231128 | |
| 100 | 1 | |a Leng, Sicong | |
| 245 | 1 | |a Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding | |
| 260 | |b Cornell University Library, arXiv.org |c Nov 28, 2023 | ||
| 513 | |a Working Paper | ||
| 520 | 3 | |a Large Vision-Language Models (LVLMs) have advanced considerably, intertwining visual recognition and language understanding to generate content that is not only coherent but also contextually attuned. Despite their success, LVLMs still suffer from the issue of object hallucinations, where models generate plausible yet incorrect outputs that include objects that do not exist in the images. To mitigate this issue, we introduce Visual Contrastive Decoding (VCD), a simple and training-free method that contrasts output distributions derived from original and distorted visual inputs. The proposed VCD effectively reduces the over-reliance on statistical bias and unimodal priors, two essential causes of object hallucinations. This adjustment ensures the generated content is closely grounded to visual inputs, resulting in contextually accurate outputs. Our experiments show that VCD, without either additional training or the usage of external tools, significantly mitigates the object hallucination issue across different LVLM families. Beyond mitigating object hallucinations, VCD also excels in general LVLM benchmarks, highlighting its wide-ranging applicability. | |
| 653 | |a Hallucinations | ||
| 653 | |a Vision | ||
| 653 | |a Training | ||
| 700 | 1 | |a Zhang, Hang | |
| 700 | 1 | |a Chen, Guanzheng | |
| 700 | 1 | |a Li, Xin | |
| 700 | 1 | |a Lu, Shijian | |
| 700 | 1 | |a Miao, Chunyan | |
| 700 | 1 | |a Bing, Lidong | |
| 773 | 0 | |t arXiv.org |g (Nov 28, 2023), p. n/a | |
| 786 | 0 | |d ProQuest |t Engineering Database | |
| 856 | 4 | 1 | |3 Citation/Abstract |u https://www.proquest.com/docview/2895042811/abstract/embedded/NVC8TPT9VN4WFQEG?source=fedsrch |
| 856 | 4 | 0 | |3 Full text outside of ProQuest |u http://arxiv.org/abs/2311.16922 |