Go to AAAI 2025 Workshop PDLM homepageBRIDO: Bringing Democratic Order to Abstractive Summarization
Keywords: abstractive summarization, factuality
TL;DR: We propose a method for mitigating hallucination through incentivizing candidates that aligns with the majority by contrastive learning.
Abstract: Hallucination refers to the inaccurate, irrelevant, and inconsistent text generated from large language models (LLMs). While the LLMs have shown great promise in a variety of tasks, the issue of hallucination still remains a major challenge for many practical uses. In this paper, we tackle the issue of hallucination in abstract text summarization by mitigating exposure bias. Existing models targeted for exposure bias mitigation, namely BRIO, aim for better summarization quality in the ROUGE score. We propose a model that uses the similar exposure bias mitigation strategy but with its goal aligned with less hallucination. We conjecture that among a group of candidate outputs, ones with hallucination will consist the minority of the whole group. Therefore candidates with less similarity with others will have a higher chance of containing hallucinated content. Our method uses this aspect and utilizes contrastive learning, incentivizing candidates with high inter-candidate ROUGE scores. We performed experiments on the XSum and CNN/DM summarization datasets, and our method showed 6.25% and 3.82% improvement, respectively, on the consistency G-Eval score over BRIO.
Submission Number: 24
Loading