pretraining

Transformer with Untied Positional Encoding (TUPE). Code of paper "Rethinking Positional Encoding in Language Pre-training". Improve existing models like BERT.

transformer language-model bert pretraining

Updated Jul 20, 2020
Python

ChandlerBang / awesome-self-supervised-gnn

Star

Papers about pretraining and self-supervised learning on Graph Neural Networks (GNN).

machine-learning deep-learning graph-mining graph-neural-networks self-supervised-learning pre-training pretraining

Updated Jul 30, 2020

surafelml / adapt-mnmt

Star

Dynamic Transfer Learning for Low-Resource Neural Machine Translation

multilingual transfer-learning nmt adaptation data-selection low-resource-languages zero-shot-translation pretraining dynamic-transfer-learning

Updated Aug 4, 2020
Python

braindeadtheory / APTOs-Blindness-Detection

Star

Pretraining on 2015, 2019 and IDRIDs with ResNet 101 and 152 and fine tuning with 2019 dataset only

deep-learning resnet aptos pretraining

Updated Dec 9, 2019
Python

arrrrrmin / albert-guide

Star

Understanding "A Lite BERT". An Transformer approach for learning self-supervised Language Models. (wip)

guide language-modeling pretrain pretraining albert-models albert-guide

Updated Mar 16, 2020
Python

chaeAclark / blog_online_pretraining

Star

This is a flexible class for training specific layers of deep neural-nets in an online manner. Supports Keras models.

machine-learning deep-neural-networks deep-learning deeplearning online-learning pretraining

Updated Aug 6, 2020
Python

Improve this page

Add a description, image, and links to the pretraining topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the pretraining topic, visit your repo's landing page and select "manage topics."

Learn more