A very good first issue IMO!

See huggingface/transformers#4829 (comment)

we should update the eval dataset to pick one start_position (or the most frequent one)

Optionally, use the huggingface/nlp library to get the eval dataset, and hook it into the Trainer.

Also referenced in huggingface/transformers#6997 (comment)

From paper, it mentioned

Instead, the training data generator chooses 15% of tokens at random, e.g., in the sentence my
dog is hairy it chooses hairy.

It means that 15% of token will be choose for sure.

From https://github.com/codertimo/BERT-pytorch/blob/master/bert_pytorch/dataset/dataset.py#L68,
for every single token, it has 15% of chance that go though the followup procedure.

Issue to track tutorial requests:

• Deep Learning with PyTorch: A 60 Minute Blitz - #69

Question
Hi, I have been experimenting with the QA capabilities of Haystack and so far. I was wondering if it was possible for the model to generate paragraph-like contexts.

Additional context
So far, when a question is asked, the model outputs an answer and the context the answer can be found in. The context output by the model is oftentimes fragments of a sentence or fragments of a para