ECD: Efficient Contrastive Decoding with Probabilistic Hallucination Detection

This chapter explores the challenges of hallucinations in Large Vision Language Models (LVLMs) and introduces Efficient Contrastive Decoding (ECD) as a solution. Hallucinations, where models generate responses inconsistent with visual inputs, are a significant issue in safety-critical applications like autonomous driving and medicine. The chapter delves into the causes of hallucinations, the limitations of existing mitigation methods, and the advantages of contrastive decoding strategies. ECD leverages probabilistic hallucination detection to suppress hallucinations during the decoding process, requiring only one forward pass of the LVLM followed by lightweight hallucination detection. This approach reduces inference time and computational overhead while improving performance across various benchmarks. The chapter also discusses the experimental setup, results, and the broader implications of ECD for enhancing the reliability and accuracy of LVLMs. Key findings include significant improvements in hallucination rates and F1 scores, demonstrating ECD's effectiveness in mitigating hallucinations and enhancing the perception capabilities of LVLMs.