Kennethborup / knowledgeDistillationLinks
PyTorch implementation of (Hinton) Knowledge Distillation and a base class for simple implementation of other distillation methods.
☆29Updated 5 years ago
Alternatives and similar repositories for knowledgeDistillation
Users that are interested in knowledgeDistillation are comparing it to the libraries listed below
Sorting:
- PyTorch, PyTorch Lightning framework for trying knowledge distillation in image classification problems☆32Updated last year
- Stochastic Weight Averaging Tutorials using pytorch.☆33Updated 5 years ago
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆39Updated 3 years ago
- A Python Package for Deep Imbalanced Learning☆56Updated 4 months ago
- ☆95Updated last year
- code for the ddp tutorial☆32Updated 3 years ago
- several types of attention modules written in PyTorch for learning purposes☆52Updated last year
- IJCAI 2021, "Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation"☆42Updated 2 years ago
- ☆109Updated 2 years ago
- Code for the PAPA paper☆27Updated 3 years ago
- Benchmarking algorithms for assessing quality of data labeled by multiple annotators☆34Updated 2 weeks ago
- The implementation of VGG thesis is implemented under PyTorch framework☆37Updated 2 years ago
- A package for fine tuning of pretrained NLP transformers using Semi Supervised Learning☆14Updated 4 years ago
- Official code for Group-Transformer (Scale down Transformer by Grouping Features for a Lightweight Character-level Language Model, COLING…☆27Updated 4 years ago
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆81Updated 2 years ago
- Code to reproduce the experiments of the ICLR24-paper: "Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging"☆12Updated 2 months ago
- Paper List for Multi-Task Learning (focus on architectures and optimization for MTL)☆51Updated 2 years ago
- Advances in Neural Information Processing Systems (NeurIPS 2021)☆22Updated 3 years ago
- AutoMoE: Neural Architecture Search for Efficient Sparsely Activated Transformers☆48Updated 3 years ago
- Adversarial examples to the new ConvNeXt architecture☆20Updated 3 years ago
- ZSKD with PyTorch☆31Updated 2 years ago
- A project to add scalable state-of-the-art out-of-distribution detection (open set recognition) support by changing two lines of code! Pe…☆79Updated 3 years ago
- The Objective is to implement Siamese Network for Face Recognition using Pytorch Lightning.☆18Updated 5 years ago
- ☆10Updated 3 years ago
- PyTorch implementation for "Few-Shot Learning with Class Imbalance"☆37Updated 4 years ago
- A project to train your model from scratch or fine-tune a pretrained model using the losses provided in this library to improve out-of-di…☆17Updated 3 years ago
- Recycling diverse models☆46Updated 2 years ago
- Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizing Long Sequences"☆70Updated 2 years ago
- Towards Understanding the Mixture-of-Experts Layer in Deep Learning☆33Updated 2 years ago
- A regularized self-labeling approach to improve the generalization and robustness of fine-tuned models☆27Updated 3 years ago