Alternatives and similar repositories for knowledgeDistillation

Users that are interested in knowledgeDistillation are comparing it to the libraries listed below

• vrvlive / knowlege-distillation
  PyTorch, PyTorch Lightning framework for trying knowledge distillation in image classification problems
  ☆32Updated last year
• jhoon-oh / kd_data
  IJCAI 2021, "Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation"
  ☆42Updated 3 years ago
• hoya012 / swa-tutorials-pytorch
  Stochastic Weight Averaging Tutorials using pytorch.
  ☆33Updated 5 years ago
• shriramsb / Distilling-the-Knowledge-in-a-Neural-Network
  Demonstration of transfer of knowledge and generalization with distillation
  ☆55Updated 7 years ago
• ntucllab / imbalanced-DL
  A Python Package for Deep Imbalanced Learning
  ☆57Updated 6 months ago
• VirtuosoResearch / Regularized-Self-Labeling
  A regularized self-labeling approach to improve the generalization and robustness of fine-tuned models
  ☆27Updated 3 years ago
• fkodom / soft-mixture-of-experts
  PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
  ☆82Updated 2 years ago
• YeonwooSung / Pytorch_mixture-of-experts
  PyTorch implementation of moe, which stands for mixture of experts
  ☆52Updated 4 years ago
• knotgrass / attention
  several types of attention modules written in PyTorch for learning purposes
  ☆53Updated last month
• wangleiofficial / label-smoothing-pytorch
  label smoothing PyTorch implementation
  ☆33Updated 5 years ago
• DefangChen / Knowledge-Distillation-Paper
  This resposity maintains a collection of important papers on knowledge distillation (awesome-knowledge-distillation)).
  ☆82Updated 10 months ago
• stanislavfort / adversaries_to_convnext
  Adversarial examples to the new ConvNeXt architecture
  ☆20Updated 4 years ago
• MAC-AutoML / YOCO-BERT
  The official implementation of You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Natu…
  ☆48Updated 4 years ago
• fgranese / DOCTOR
  Advances in Neural Information Processing Systems (NeurIPS 2021)
  ☆22Updated 3 years ago
• da2so / Zero-shot_Knowledge_Distillation_Pytorch
  ZSKD with PyTorch
  ☆31Updated 2 years ago
• VITA-Group / Nasty-Teacher
  [ICLR 2021 Spotlight Oral] "Undistillable: Making A Nasty Teacher That CANNOT teach students", Haoyu Ma, Tianlong Chen, Ting-Kuei Hu, Che…
  ☆82Updated 4 years ago
• Pengxin-Guo / Awesome-Multi-Task-Learning
  Paper List for Multi-Task Learning (focus on architectures and optimization for MTL)
  ☆51Updated 2 years ago
• lucidrains / tableformer-pytorch
  Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch
  ☆39Updated 3 years ago
• mattochal / imbalanced_fsl_public
  PyTorch implementation for "Few-Shot Learning with Class Imbalance"
  ☆37Updated 4 years ago
• The-AI-Summer / pytorch-ddp
  code for the ddp tutorial
  ☆32Updated 3 years ago
• ShivamRajSharma / Transformer-Architectures-From-Scratch
  Implementation of transformers based architecture in PyTorch.
  ☆55Updated 5 years ago
• huawei-noah / Efficient-NLP
  ☆95Updated last year
• Lornatang / VGG-PyTorch
  The implementation of VGG thesis is implemented under PyTorch framework
  ☆38Updated 3 years ago
• mansheej / data_diet
  ☆111Updated 2 years ago
• facebookresearch / ModelRatatouille
  Recycling diverse models
  ☆46Updated 3 years ago
• cleanlab / multiannotator-benchmarks
  Benchmarking algorithms for assessing quality of data labeled by multiple annotators
  ☆34Updated 2 months ago
• JamesLYC88 / text_classification_baseline_code
  The code for the ACL 2023 paper "Linear Classifier: An Often-Forgotten Baseline for Text Classification".
  ☆19Updated last year
• clovaai / attention-feature-distillation
  Official implementation for (Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching, AAAI-2021)
  ☆119Updated 4 years ago
• sayakpaul / robustness-vit
  Contains code for the paper "Vision Transformers are Robust Learners" (AAAI 2022).
  ☆122Updated 3 years ago
• raymin0223 / self-contrastive-learning
  Self-Contrastive Learning: Single-viewed Supervised Contrastive Framework using Sub-network (AAAI 2023)
  ☆21Updated 2 years ago