Knowledge-Distillation

• ColBERTv2

  Improved ColBERT with cross-encoder distillation and residual compression; dramatically reduces index size while matching or exceeding v1 effectiveness.

• Knowledge Distillation for IR

  Training a fast bi-encoder (student) to mimic the ranking scores of a slow cross-encoder (teacher); the dominant approach for improving dense retrieval without cross-encoder latency.

• TAS-B

  Topic-Aware Sampling with BERT; dense retrieval model trained via balanced topic-aware sampling and cross-encoder distillation, achieving strong recall with efficient inference.