Training

• Contrastive Loss

  Training objective that pulls similar pairs together and pushes dissimilar pairs apart in embedding space; the dominant loss function for dense retrieval and sentence embedding models.

• Hard Negative Mining

  Strategy for selecting training negatives that are difficult for the current model to distinguish from positives; critical for dense retrieval model quality beyond in-batch random negatives.

• In-Batch Negatives

  Training technique where other (query, passage) pairs within the same mini-batch serve as negatives; free negative supervision that scales with batch size.

• Knowledge Distillation for IR

  Training a fast bi-encoder (student) to mimic the ranking scores of a slow cross-encoder (teacher); the dominant approach for improving dense retrieval without cross-encoder latency.