Kennethborup / knowledgeDistillation
PyTorch implementation of (Hinton) Knowledge Distillation and a base class for simple implementation of other distillation methods.
☆28Updated 4 years ago
Alternatives and similar repositories for knowledgeDistillation
Users that are interested in knowledgeDistillation are comparing it to the libraries listed below
Sorting:
- IJCAI 2021, "Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation"☆41Updated 2 years ago
- ZSKD with PyTorch☆30Updated last year
- Data-free knowledge distillation using Gaussian noise (NeurIPS paper)☆15Updated 2 years ago
- PyTorch, PyTorch Lightning framework for trying knowledge distillation in image classification problems☆32Updated 9 months ago
- This resposity maintains a collection of important papers on knowledge distillation (awesome-knowledge-distillation)).☆77Updated last month
- Code for the PAPA paper☆27Updated 2 years ago
- Demonstration of transfer of knowledge and generalization with distillation☆53Updated 6 years ago
- Source code for IJCAI 2022 Long paper: Parameter-Efficient Sparsity for Large Language Models Fine-Tuning.☆14Updated 2 years ago
- [ICLR 2021 Spotlight Oral] "Undistillable: Making A Nasty Teacher That CANNOT teach students", Haoyu Ma, Tianlong Chen, Ting-Kuei Hu, Che…☆81Updated 3 years ago
- Code for "Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources". (IC…☆38Updated 4 years ago
- [ICLR 2022] "Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, and No Retraining" by Lu Miao*, Xiaolong Luo*, T…☆29Updated 3 years ago
- Stochastic Weight Averaging Tutorials using pytorch.☆33Updated 4 years ago
- A generic code base for neural network pruning, especially for pruning at initialization.☆30Updated 2 years ago
- Repo to reproduce results for Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning☆25Updated 2 years ago
- AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers☆46Updated 2 years ago
- A regularized self-labeling approach to improve the generalization and robustness of fine-tuned models☆28Updated 2 years ago
- Unofficial Pytorch Implementation Of AdversarialAutoAugment(ICLR2020)☆21Updated 4 years ago
- Pytorch implementation of our paper accepted by IEEE TNNLS, 2022 -- Distilling a Powerful Student Model via Online Knowledge Distillation☆28Updated 3 years ago
- ☆21Updated 2 years ago
- A simple reimplement Online Knowledge Distillation via Collaborative Learning with pytorch☆48Updated 2 years ago
- PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Repr…☆22Updated 3 years ago
- Code for Active Mixup in 2020 CVPR☆22Updated 3 years ago
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆24Updated 2 years ago
- sharpDARTS: Faster and More Accurate Differentiable Architecture Search☆16Updated 4 years ago
- ☆10Updated 3 years ago
- Pytorch implementations of Co-teaching for noisy label learning☆13Updated 2 years ago
- code for the ddp tutorial☆32Updated 3 years ago
- Official code for Group-Transformer (Scale down Transformer by Grouping Features for a Lightweight Character-level Language Model, COLING…☆25Updated 4 years ago
- [NeurIPS‘2021] "MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge", Geng Yuan, Xiaolong Ma, Yanzhi Wang et al…☆18Updated 3 years ago
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆60Updated 3 years ago