Pretraining
-
REALM
Retrieval-Augmented Language Model Pretraining; jointly trains a retriever and language model by backpropagating through retrieval during masked language modeling pretraining.
Retrieval-Augmented Language Model Pretraining; jointly trains a retriever and language model by backpropagating through retrieval during masked language modeling pretraining.