Cloze-Task

• Cloze Task

  Predicting masked tokens from context; unsupervised pre-training objective where random words are hidden and must be inferred.