distillation

Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller

Awesome Knowledge Distillation

Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。

A PyTorch-based knowledge distillation toolkit for natural language processing

PaddleSlim is an open-source library for deep model compression and architecture search.

Pytorch implementation of various Knowledge Distillation (KD) methods.

PyTorch implementation of various methods for continual learning (XdG, EWC, SI, LwF, FROMP, DGR, BI-R, ER, A-GEM, iCaRL, Generative Classifier) in three different scenarios.

高质量中文预训练模型集合：最先进大模型、最快小模型、相似度专门模型

mobilev2-yolov5s剪枝、蒸馏，支持ncnn，tensorRT部署。ultra-light but better performence！

MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks

A Python library for adversarial machine learning focusing on benchmarking adversarial robustness.

(CVPR 2022) A minimalist, mapless, end-to-end self-driving stack for joint perception, prediction, planning and control.

Quantization library for PyTorch. Support low-precision and mixed-precision quantization, with hardware implementation through TVM.

BERT distillation（基于BERT的蒸馏实验 ）

推荐/广告/搜索领域工业界经典以及最前沿论文集合。A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.

A brain-inspired version of generative replay for continual learning with deep neural networks (e.g., class-incremental learning on CIFAR-100; PyTorch code).

Knowledge distillation in text classification with pytorch. 知识蒸馏，中文文本分类，教师模型BERT、XLNET，学生模型biLSTM。

Papers and Book to look at when starting NLP 📚

Insightface Keras implementation

Distillation of KoBERT from SKTBrain (Lightweight KoBERT)