Jina-ColBERT-v2: A General-Purpose Multilingual Late Interaction
  Retriever

Rohan Jha; Bo Wang; Michael Günther; Saba Sturua; Mohammad Kalim Akram; Han Xiao

返回文献探索

Paper · arXiv 2408.16672

Jina-ColBERT-v2: A General-Purpose Multilingual Late Interaction Retriever

Rohan Jha, Bo Wang, Michael Günther, Saba Sturua, Mohammad Kalim Akram, Han Xiao

10 upvotesAugust 29, 2024arXiv 预印本

AI 摘要

Jina-ColBERT-v2 enhances ColBERT's performance and reduces storage requirements through improvements to its architecture and training pipeline, benefiting multilingual retrieval tasks.

multi-vector dense modelsColBERTlate interaction scoringjoint query-document attentioncross-encodersbi-encoder architectureindexingsearchsingle-vector embeddingJina-ColBERT-v2heterogeneous multilingual data

Abstract

Multi-vector dense models, such as ColBERT, have proven highly effective in information retrieval. ColBERT's late interaction scoring approximates the joint query-document attention seen in cross-encoders while maintaining inference efficiency closer to traditional dense retrieval models, thanks to its bi-encoder architecture and recent optimizations in indexing and search. In this paper, we introduce several improvements to the ColBERT model architecture and training pipeline, leveraging techniques successful in the more established single-vector embedding model paradigm, particularly those suited for heterogeneous multilingual data. Our new model, Jina-ColBERT-v2, demonstrates strong performance across a range of English and multilingual retrieval tasks, while also cutting storage requirements by up to 50% compared to previous models.