🚀 Feature request

modeling_longformer.py has the classes LongformerForSequenceClassification, LongformerForMultipleChoice and LongformerForTokenClassification which are not present in modeling_tf_longformer.py at the moment.
Those classes should be equally added to modeling_tf_longformer.py.

Motivation

The pretrained weights for TFLongformer are available so that these

From paper, it mentioned

Instead, the training data generator chooses 15% of tokens at random, e.g., in the sentence my
dog is hairy it chooses hairy.

It means that 15% of token will be choose for sure.

From https://github.com/codertimo/BERT-pytorch/blob/master/bert_pytorch/dataset/dataset.py#L68,
for every single token, it has 15% of chance that go though the followup procedure.

Describe the feature

I think enforcing typing in methods parameters can be helpful for robustness, readability and stability of the code.

By using mypy static type checker, we can see potential improvements for jina:

Usage:

pip install mypy
mypy --ignore-missing-imports jina

Do not get overwhelmed by the errors. Let's slowly keep improving until we can eve

If I want to use both of them, how to modify code in aen.py? Thanks a lot.

Ideally, we'd support something like mnli += {pretrain_data_fraction = 0.5}, pretrain_tasks = {mnli,boolq}. Currently, pretrain_data_fraction is just a global argument for all pretraining tasks.