sutd-visual-computing-group / LS-KD-compatibility
[ICML 2022] This work investigates the compatibility between label smoothing (LS) and knowledge distillation (KD). We suggest to use an LS-trained teacher with a low-temperature transfer to render high performance students.
☆11Updated 2 years ago
Alternatives and similar repositories for LS-KD-compatibility:
Users that are interested in LS-KD-compatibility are comparing it to the libraries listed below
- Information Bottleneck Approach to Spatial Attention Learning, IJCAI2021☆14Updated 3 years ago
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆23Updated last year
- The official project website of "NORM: Knowledge Distillation via N-to-One Representation Matching" (The paper of NORM is published in IC…☆19Updated last year
- Learning from Limited and Imperfect Data (L2ID): Classification Challenges☆18Updated 3 years ago
- Source code for the BMVC-2021 paper "SimReg: Regression as a Simple Yet Effective Tool for Self-supervised Knowledge Distillation".☆16Updated 3 years ago
- ☆19Updated 3 years ago
- Prior Knowledge Guided Unsupervised Domain Adaptation (ECCV 2022)☆16Updated 2 years ago
- [WACV2023] This is the official PyTorch impelementation of our paper "[Rethinking Rotation in Self-Supervised Contrastive Learning: Adapt…☆12Updated last year
- Dual Adaptive Representation Alignment for Cross-domain Few-shot Learning TPAMI 2023☆21Updated 8 months ago
- TF-FD☆19Updated 2 years ago
- 🔥MixPro: Data Augmentation with MaskMix and Progressive Attention Labeling for Vision Transformer [Official, ICLR 2023]☆20Updated last year
- Pytorch implementation (TPAMI 2023) - Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion☆19Updated 2 years ago
- This repository contains the code for our AAAI2021 paper CAKES: Channel-wise Automatic KErnel Shrinking for Efficient 3D Networks.☆12Updated 4 years ago
- [ICLR 2022]: Fast AdvProp☆34Updated 2 years ago
- NeurIPS 2022: Estimating Noise Transition Matrix with Label Correlations for Noisy Multi-Label Learning☆17Updated last year
- Black-box Few-shot Knowledge Distillation☆11Updated 2 years ago
- [ICASSP 2020] Code release of paper 'Heterogeneous Domain Generalization via Domain Mixup'☆24Updated 4 years ago
- This is the implementation of our CVPR'23 paper "Class-Conditional Sharpness-Aware Minimization for Deep Long-Tailed Recognition".☆17Updated last year
- ☆19Updated 2 years ago
- ☆14Updated 2 years ago
- ResMLP: Feedforward networks for image classification with data-efficient training☆42Updated 3 years ago
- The official code repo for the paper "Mixture of Stochastic Experts for Modeling Aleatoric Uncertainty in Segmentation". (ICLR 2023)☆26Updated last year
- ☆29Updated last year
- ☆12Updated 2 years ago
- ☆29Updated 3 years ago
- ☆16Updated last year
- ☆21Updated 2 years ago
- [CVPR 2022] "The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy" by Tianlong C…☆25Updated 2 years ago
- Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimization (IEEE TPAMI 2021)☆17Updated 3 years ago
- PyTorch Implementation of "Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model"☆48Updated 2 years ago