Trusted Source Alignment in Large Language Models

Vasilisa Bashlovkina; Zhaobin Kuang; Riley Matthews; Edward Clifford; Yennie Jun; William W. Cohen; Simon Baumgartner

返回文献探索

Paper · arXiv 2311.06697

Trusted Source Alignment in Large Language Models

Vasilisa Bashlovkina, Zhaobin Kuang, Riley Matthews, Edward Clifford, Yennie Jun, William W. Cohen, Simon Baumgartner

11 upvotesNovember 12, 2023arXiv 预印本

AI 摘要

A new evaluation metric called trusted source alignment (TSA) is introduced and measured for large language models to assess their preference for content from trusted publishers over contradicting information.

large language models (LLMs)trusted source alignment (TSA)fact checking articlesresponse extractionclaim contextualizationprompt formulationPaLM-2balanced accuracy

Abstract

Large language models (LLMs) are trained on web-scale corpora that inevitably include contradictory factual information from sources of varying reliability. In this paper, we propose measuring an LLM property called trusted source alignment (TSA): the model's propensity to align with content produced by trusted publishers in the face of uncertainty or controversy. We present FactCheckQA, a TSA evaluation dataset based on a corpus of fact checking articles. We describe a simple protocol for evaluating TSA and offer a detailed analysis of design considerations including response extraction, claim contextualization, and bias in prompt formulation. Applying the protocol to PaLM-2, we find that as we scale up the model size, the model performance on FactCheckQA improves from near-random to up to 80% balanced accuracy in aligning with trusted sources.