Zhanghao Hu, Qinglin Zhu, Siya Qi +3 authors
Large Language Models (LLMs) have shown improved generation performance
through retrieval-augmented generation (RAG) following the retriever-reader
paradigm, which supplements model inputs with externally retrieved knowledge.
However, prior work often evaluates RAG holistically, assessing the retriever
and reader jointly, making it difficult to isolate the true contribution of
retrieval, particularly given the prompt sensitivity of LLMs used as readers.
We introduce Spectrum Projection Score (SPS), a lightweight, supervision-free
metric that allows the reader to gauge the semantic alignment of a retrieved
summary with its hidden representation by comparing the area formed by
generated tokens from the summary, and the principal directions of subspace in
the reader and to measure the relevance. Building on SPS we present xCompress,
an inference time controller framework that dynamically samples, ranks, and
compresses retrieval summary candidates. Extensive experiments on five QA
benchmarks with four open source LLMs show that SPS not only enhances
performance across a range of tasks but also provides a principled perspective
on the interaction between retrieval and generation.