language-model

Feature request

Is the addition of the 'OPTforSequenceClassification' class scheduled?
Is someone handling it?
When adding these functions, I wonder if it is possible to PR one by one, or if I have to PR all classes supported by other models.

Motivation

Added function of OPT class, which is being actively discussed recently

Your contribution

I personally use the forSequenceCla

From paper, it mentioned

Instead, the training data generator chooses 15% of tokens at random, e.g., in the sentence my
dog is hairy it chooses hairy.

It means that 15% of token will be choose for sure.

From https://github.com/codertimo/BERT-pytorch/blob/master/bert_pytorch/dataset/dataset.py#L68,
for every single token, it has 15% of chance that go though the followup procedure.

When users run our tutorial notebooks, there are quite many convoluted log messages.

For example, there is the log message regarding apex and others:

"INFO - haystack.document_stores.base -  Numba not found, replacing njit() with no-op implementation. Enable it with 'pip install numba'.\n",
"INFO - haystack.modeling.model.optimization -  apex not found, won't use it. See https://nvidia.g

Describe the bug
Setting "text-gen-type": "interactive" results in an IndexError: : shape mismatch: indexing tensors could not be broadcast together with shapes [4], [3]. Other generation types work.

To Reproduce
Steps to reproduce the behavior:

1. Install, adapt 20B to local environment, add "text-gen-type": "interactive" config
2. Run inference
3. Enter arbitrary prompt when

Issue to track tutorial requests:

• Deep Learning with PyTorch: A 60 Minute Blitz - #69
• Sentence Classification - #79

I've been chatting with some others interested in training CLIP for different domain tasks. They expressed interest in a simple way to use a pre-trained text transformer.

Some basic support for Hugging Face or generic classes of transformers shouldn't be too crazy of an extension to what is already fleshed out.