ColBERT v2

ColBERTv2 is a retrieval model that uses contextualized late interaction over BERT to enable fast and accurate search over large text collections. It encodes each passage into a matrix of token-level embeddings and each query into another matrix, then efficiently scores fine-grained similarity between them using scalable vector-similarity (MaxSim) operators. This approach allows ColBERTv2 to surpass the quality of single-vector representation models while scaling efficiently.

Key Strengths

• Fine-grained contextual late interaction for high retrieval accuracy.
• Pre-computed document representations enable offline indexing and fast query processing.
• ColBERTv2 reduces the space footprint of late interaction models by 6–10x compared to vanilla late interaction.
• PLAID engine speeds up vanilla ColBERTv2 search latency by up to 7x on GPU and 45x on CPU while maintaining state-of-the-art retrieval quality.
• Achieves latency of tens of milliseconds on GPU and tens to few hundred milliseconds on CPU at large scale (up to 140M passages).

Benchmark Results

ColBERTv2 has been evaluated on the LoTTE benchmark test set, yielding the following Success@5 scores:

Topic	Search Queries	Forum Queries
Writing	80.1	76.3
Recreation	72.3	70.8
Science	56.7	46.1
Technology	66.1	53.6
Lifestyle	84.7	76.9
Pooled	71.6	63.4

ColBERTv2 is competitive with existing BERT-based models and outperforms every non-BERT baseline, while executing two orders-of-magnitude faster and requiring four orders-of-magnitude fewer FLOPs per query. It has also been adapted for Open-Domain Question Answering (ColBERT-QA), attaining state-of-the-art extractive OpenQA performance on Natural Questions, SQuAD, and TriviaQA.