Awesome Knowledge Distillation
-
Updated
May 30, 2021
#
knowledge-distillation
Here are 159 public repositories matching this topic...
A treasure chest for visual recognition powered by PaddlePaddle
image-classification
image-recognition
pretrained-models
knowledge-distillation
product-recognition
autoaugment
cutmix
randaugment
gridmask
deit
repvgg
swin-transformer
image-retrieval-system
-
Updated
Aug 11, 2021 - Python
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
-
Updated
Jul 27, 2021 - Python
NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego
natural-language-processing
deep-learning
text-classification
dnn
pytorch
artificial-intelligence
question-answering
knowledge-distillation
sequence-labeling
text-matching
qna
model-compression
-
Updated
Mar 16, 2021 - Python
Improving Convolutional Networks via Attention Transfer (ICLR 2017)
-
Updated
Jul 11, 2018 - Jupyter Notebook
A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
deep-neural-networks
computer-vision
pytorch
knowledge-distillation
cifar10
dark-knowledge
model-compression
-
Updated
Jun 8, 2021 - Python
PaddleSlim is an open-source library for deep model compression and architecture search.
pruning
quantization
nas
knowledge-distillation
evolution-strategy
model-compression
neural-architecture-search
hyperparameter-search
autodl
-
Updated
Aug 11, 2021 - Python
Pytorch implementation of various Knowledge Distillation (KD) methods.
knowledge-distillation
teacher-student
knowledge-transfer
model-compression
distillation
kd
kd-methods
-
Updated
Jun 11, 2021 - Python
EasyTransfer is designed to make the development of transfer learning in NLP applications easier.
-
Updated
Jul 30, 2021 - Python
利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码
flask
deployment
random-forest
svm
cnn
pytorch
image-classification
densenet
resnet
knn
knowledge-distillation
resnext
label-smoothing
-
Updated
Apr 13, 2021 - Jupyter Notebook
knowledge distillation papers
-
Updated
Jun 9, 2020
Data Efficient Model Compression
-
Updated
Jul 12, 2021 - Python
Knowledge Distillation: CVPR2020 Oral, Revisiting Knowledge Distillation via Label Smoothing Regularization
-
Updated
Jun 2, 2021 - Python
PyTorch-based modular, configuration-driven framework for knowledge distillation. 🏆 19 methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
nlp
natural-language-processing
glue
pytorch
transformer
imagenet
image-classification
coco
object-detection
semantic-segmentation
knowledge-distillation
cifar10
pascal-voc
cifar100
google-colab
colab-notebook
-
Updated
Aug 3, 2021 - Python
Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)
-
Updated
Jun 23, 2020 - Python
Official pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
deep-learning
human-pose-estimation
mpii-dataset
knowledge-distillation
mscoco-keypoint
coco-keypoints-detection
fast-pose-distillation
-
Updated
Mar 27, 2020 - Cuda
Pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
-
Updated
Jun 8, 2021 - Python
(MLSys' 21) An Acceleration System for Large-scare Unsupervised Heterogeneous Outlier Detection (Anomaly Detection)
python
distributed-systems
machine-learning
data-mining
machine-learning-algorithms
machine-learning-library
outlier-detection
knowledge-distillation
anomaly-detection
-
Updated
Jan 19, 2021 - Python
Official pytorch Implementation of Relational Knowledge Distillation, CVPR 2019
-
Updated
May 17, 2021 - Python
Infrastructures™ for Machine Learning Training/Inference in Production.
kubernetes
machine-learning
apache-spark
deep-learning
artificial-intelligence
awesome-list
pruning
quantization
knowledge-distillation
deep-learning-framework
model-compression
apache-arrow
federated-learning
machine-learning-systems
apache-mesos
-
Updated
May 24, 2019
Knowledge distillation methods implemented with Tensorflow (now there are 11 (+1) methods, and will be added more.)
-
Updated
Nov 21, 2019 - Python
Using Teacher Assistants to Improve Knowledge Distillation: https://arxiv.org/pdf/1902.03393.pdf
-
Updated
Oct 3, 2019 - Python
A large scale study of Knowledge Distillation.
-
Updated
Apr 19, 2020 - Python
[CVPR'20] PyTorch code for our paper "Collaborative Distillation for Ultra-Resolution Universal Style Transfer"
-
Updated
Aug 6, 2021 - Python
Paper Lists, Notes and Slides, Focus on NLP. For summarization, please refer to https://github.com/xcfcode/Summarization-Papers
nlp
naacl
notes
presentation
slides
acl
presentations
conversation
gan
generation
summarization
emnlp
knowledge-distillation
data-augmentation
aaai
meta-learning
graph-neural-networks
gnn
pretrain
non-autoregressive
-
Updated
Aug 11, 2021
Knowledge Distillation using Tensorflow
-
Updated
Aug 12, 2019 - Python
MicroExpNet: An Extremely Small and Fast Model For Expression Recognition From Frontal Face Images
machine-learning
deep-neural-networks
deep-learning
tensorflow
knowledge-distillation
facial-expression-recognition
model-compression
-
Updated
Mar 7, 2021 - Python
Training & evaluation library for text-based neural re-ranking and dense retrieval models built with PyTorch
-
Updated
Jul 26, 2021 - Python
Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons (AAAI 2019)
transfer-learning
knowledge-distillation
knowledge-transfer
network-compression
teacher-student-learning
-
Updated
Sep 9, 2019 - Python
Improve this page
Add a description, image, and links to the knowledge-distillation topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the knowledge-distillation topic, visit your repo's landing page and select "manage topics."
We also need to benchmark the Lottery-tickets Pruning algorithm and the Quantization algorithms. The models used for this would be the student networks discussed in #105 (ResNet18, MobileNet v2, Quantization v2).
Pruning (benchmark upto 40, 50 and 60 % pruned weights)
Quantization