Go to https://github.com/pytorch/tutorials - this repo is deprecated and no longer maintained
-
Updated
Jul 1, 2021 - Jupyter Notebook
Add a description, image, and links to the nlg topic page so that developers can more easily learn about it.
To associate your repository with the nlg topic, visit your repo's landing page and select "manage topics."
Description
While using tokenizers.create with the model and vocab file for a custom corpus, the code throws an error and is not able to generate the BERT vocab file
Error Message
ValueError: Mismatch vocabulary! All special tokens specified must be control tokens in the sentencepiece vocabulary.
To Reproduce
from gluonnlp.data import tokenizers
tokenizers.create('spm', model_p