sutd-visual-computing-group / LS-KD-compatibility
[ICML 2022] This work investigates the compatibility between label smoothing (LS) and knowledge distillation (KD). We suggest to use an LS-trained teacher with a low-temperature transfer to render high performance students.
☆11Updated last year
Related projects: ⓘ
- The official project website of "NORM: Knowledge Distillation via N-to-One Representation Matching" (The paper of NORM is published in IC…☆19Updated last year
- Switchable Online Knowledge Distillation☆16Updated last year
- Information Bottleneck Approach to Spatial Attention Learning, IJCAI2021☆14Updated 3 years ago
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆23Updated last year
- TF-FD☆20Updated last year
- [ICLR 2022]: Fast AdvProp☆35Updated 2 years ago
- Pytorch implementation (TPAMI 2023) - Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion☆17Updated last year
- This repo is the official megengine implementation of the ECCV2022 paper: Efficient One Pass Self-distillation with Zipf's Label Smoothin…☆25Updated last year
- ☆20Updated 2 years ago
- Prior Knowledge Guided Unsupervised Domain Adaptation (ECCV 2022)☆17Updated 2 years ago
- The offical implement of ImbSAM (Imbalanced-SAM)☆21Updated 6 months ago
- ☆26Updated last year
- A generic code base for neural network pruning, especially for pruning at initialization.☆30Updated 2 years ago
- Learning from Limited and Imperfect Data (L2ID): Classification Challenges☆16Updated 3 years ago
- [Preprint] Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Prunin…☆40Updated last year
- [NeurIPS'22] What Makes a "Good" Data Augmentation in Knowledge Distillation -- A Statistical Perspective☆35Updated last year
- ☆11Updated 2 years ago
- Learning with Noisy Labels, Label Noise, ICML 2021☆38Updated last year
- [ICASSP 2020] Code release of paper 'Heterogeneous Domain Generalization via Domain Mixup'☆24Updated 4 years ago
- Repo for the paper "Extrapolating from a Single Image to a Thousand Classes using Distillation"☆37Updated 2 months ago
- [CVPR 2022] "The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy" by Tianlong C…☆25Updated 2 years ago
- Data-free knowledge distillation using Gaussian noise (NeurIPS paper)☆15Updated last year
- This is an official implementation of our CVPR 2020 paper "Non-Local Neural Networks With Grouped Bilinear Attentional Transforms".☆12Updated 3 years ago
- ☆16Updated last year
- The reproduce for "AM-LFS: AutoML for Loss Function Search"☆13Updated 4 years ago
- Source code for the BMVC-2021 paper "SimReg: Regression as a Simple Yet Effective Tool for Self-supervised Knowledge Distillation".☆17Updated 2 years ago
- NeurIPS 2022: Estimating Noise Transition Matrix with Label Correlations for Noisy Multi-Label Learning☆16Updated last year
- Code implementation for paper "On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals".☆16Updated 2 years ago
- Official implementation of the paper "Function-Consistent Feature Distillation" (ICLR 2023)☆25Updated last year
- ☆19Updated last year