sutd-visual-computing-group / LS-KD-compatibility
[ICML 2022] This work investigates the compatibility between label smoothing (LS) and knowledge distillation (KD). We suggest to use an LS-trained teacher with a low-temperature transfer to render high performance students.
☆11Updated last year
Related projects ⓘ
Alternatives and complementary repositories for LS-KD-compatibility
- [WACV2023] This is the official PyTorch impelementation of our paper "[Rethinking Rotation in Self-Supervised Contrastive Learning: Adapt…☆12Updated last year
- The official project website of "NORM: Knowledge Distillation via N-to-One Representation Matching" (The paper of NORM is published in IC …☆19Updated last year
- Pytorch implementation (TPAMI 2023) - Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion☆19Updated last year
- ☆14Updated 2 years ago
- TF-FD☆20Updated 2 years ago
- NeurIPS 2022: Estimating Noise Transition Matrix with Label Correlations for Noisy Multi-Label Learning☆17Updated last year
- Source code for the BMVC-2021 paper "SimReg: Regression as a Simple Yet Effective Tool for Self-supervised Knowledge Distillation".☆17Updated 2 years ago
- Learning from Limited and Imperfect Data (L2ID): Classification Challenges☆17Updated 3 years ago
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆23Updated last year
- Official Repository for Soft Augmentation☆18Updated last year
- [CVPR 2022] "The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy" by Tianlong C…☆25Updated 2 years ago
- Information Bottleneck Approach to Spatial Attention Learning, IJCAI2021☆14Updated 3 years ago
- The offical implement of ImbSAM (Imbalanced-SAM)☆21Updated 8 months ago
- [ICLR 2022]: Fast AdvProp☆35Updated 2 years ago
- Code for the ICML 2021 paper "Sharing Less is More: Lifelong Learning in Deep Networks with Selective Layer Transfer"☆11Updated 3 years ago
- [Preprint] Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Prunin…☆40Updated last year
- Black-box Few-shot Knowledge Distillation☆11Updated 2 years ago
- Switchable Online Knowledge Distillation☆17Updated 3 weeks ago
- Code implementation for paper "On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals".☆16Updated 2 years ago
- A generic code base for neural network pruning, especially for pruning at initialization.☆30Updated 2 years ago
- IJCAI 2021, "Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation"☆39Updated last year
- ☆10Updated 6 months ago
- ☆20Updated 3 years ago
- The first contrastive learning work for Active Learning.☆15Updated last year
- ☆14Updated 6 months ago
- 🔥MixPro: Data Augmentation with MaskMix and Progressive Attention Labeling for Vision Transformer [Official, ICLR 2023]☆20Updated last year
- Code for Paper "Self-Distillation from the Last Mini-Batch for Consistency Regularization"☆41Updated 2 years ago
- Prior Knowledge Guided Unsupervised Domain Adaptation (ECCV 2022)☆16Updated 2 years ago
- ☆19Updated 3 years ago
- [NeurIPS'22] What Makes a "Good" Data Augmentation in Knowledge Distillation -- A Statistical Perspective☆36Updated last year