Dense-Retrieval

• Retrieval-Augmented Generation

  Grounding language model generation in retrieved external documents; reduces hallucination and enables knowledge updates without retraining.

• Hybrid Search

  Combining dense vector similarity and sparse term-matching scores to balance semantic understanding with keyword precision.

• ANCE

  Approximate Nearest Neighbor Negative Contrastive Estimation; improves dense retrieval training by dynamically refreshing hard negatives from the current model’s ANN index.

• ColBERTv2

  Improved ColBERT with cross-encoder distillation and residual compression; dramatically reduces index size while matching or exceeding v1 effectiveness.

• Contriever

  Unsupervised dense retrieval model trained with contrastive learning on unlabeled text; no labeled query-passage pairs required.

• DPR (Dense Passage Retrieval)

  Dual BERT encoder model that retrieves passages by embedding queries and documents into a shared dense vector space; foundational bi-encoder for open-domain QA.

• DSSM

  Deep Structured Semantic Model (2013); the original neural dual-encoder for web search, using word-hash trigram inputs and MLP towers to learn query-document semantic similarity.

• Hard Negative Mining

  Strategy for selecting training negatives that are difficult for the current model to distinguish from positives; critical for dense retrieval model quality beyond in-batch random negatives.

• HyDE

  Hypothetical Document Embeddings; generates a hypothetical answer to a query using an LLM and embeds that instead of the original query for zero-shot dense retrieval improvement.

• Query2Doc

  Expands queries by prepending LLM-generated pseudo-documents before retrieval; improves both sparse and dense retrieval without modifying the index or retrieval model.

• SimCSE

  Simple Contrastive Sentence Embeddings; learns high-quality sentence representations via dropout-based augmentation (unsupervised) or NLI entailment pairs (supervised).

• TAS-B

  Topic-Aware Sampling with BERT; dense retrieval model trained via balanced topic-aware sampling and cross-encoder distillation, achieving strong recall with efficient inference.

• ColBERT

  Contextualized Late Interaction over BERT; late-interaction ranking using per-token embeddings with MaxSim scoring for efficient dense retrieval.

• Bi-Encoder

  Neural architecture encoding query and document independently into separate embeddings, enabling fast retrieval via approximate nearest-neighbour search.

• Approximate Nearest Neighbour

  Fast nearest-neighbour search algorithm sacrificing exactness for speed; enables practical dense retrieval at scale. Abbreviated ANN.

• Comparing BM25 and Dense Retrieval for a Product Catalogue

  A side-by-side evaluation of keyword search and embedding-based search on a realistic product dataset, showing where each approach wins and how hybrid search splits the difference.