Retrieval-Augmented

• Atlas

  Few-shot retrieval-augmented language model combining FiD reader with Contriever retriever; jointly fine-tuned to achieve strong few-shot performance on knowledge-intensive tasks.

• FiD (Fusion-in-Decoder)

  Encodes each retrieved passage independently with T5, then fuses all passage representations in the decoder; more scalable than concatenating all passages as a single long input.

• REALM

  Retrieval-Augmented Language Model Pretraining; jointly trains a retriever and language model by backpropagating through retrieval during masked language modeling pretraining.