ZuchniakK / MTKDLinks
Multi-Teacher Knowledge Distillation, code for my PhD dissertation. I used knowledge distillation as a decision-fusion and compressing mechanism for ensemble models.
☆21Updated 2 years ago
Alternatives and similar repositories for MTKD
Users that are interested in MTKD are comparing it to the libraries listed below
Sorting:
- This is the implementation for the ICME-2023 paper (Adaptive Multi-Teacher Knowledge Distillation with Meta-Learning).☆27Updated 2 years ago
- Wavelet-Attention CNN for Image Classification☆29Updated 3 years ago
- This resposity maintains a collection of important papers on knowledge distillation (awesome-knowledge-distillation)).☆79Updated 3 months ago
- [NeurIPS2023]Lightweight Vision Transformer with Bidirectional Interaction☆24Updated last year
- AMTML-KD: Adaptive Multi-teacher Multi-level Knowledge Distillation☆60Updated 4 years ago
- Multi-head Recurrent Layer Attention for Vision Network☆19Updated 2 years ago
- Elsevier Templates-Latex☆63Updated 2 months ago
- The official repo for CVPR2023 highlight paper "Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization".☆83Updated 2 years ago
- PolyLoss implementation using PyTorch☆12Updated 3 years ago
- Official code release of our paper "EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attention"☆21Updated 9 months ago
- AAAI 2022 papers with code☆36Updated 3 years ago
- Code for "Dual Focal Loss for Calibration" (ICML 2023)☆31Updated 2 months ago
- Training ImageNet / CIFAR models with sota strategies and fancy techniques such as ViT, KD, Rep, etc.☆83Updated last year
- [ICML 2022] This work investigates the compatibility between label smoothing (LS) and knowledge distillation (KD). We suggest to use an L…☆11Updated 2 years ago
- AdaTask: A Task-Aware Adaptive Learning Rate Approach to Multi-Task Learning. AAAI, 2023.☆28Updated last year
- Official implementation of Bayes Conditional Distribution Estimation for Knowledge Distillation Based on Conditional Mutual Information☆11Updated last year
- Official implementation of paper "Knowledge Distillation from A Stronger Teacher", NeurIPS 2022☆147Updated 2 years ago
- EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm☆34Updated 2 years ago
- Trainable Highly-expressive Activation Functions. ECCV 2024☆37Updated 4 months ago
- Official PyTorch(MMCV) implementation of “Adversarial AutoMixup” (ICLR 2024 spotlight)☆69Updated 8 months ago
- Official implementation of the paper "Masked Autoencoders are Efficient Class Incremental Learners"☆42Updated last year
- BM-NAS: Bilevel Multimodal Neural Architecture Search (AAAI 2022 Oral)☆18Updated 2 years ago
- Github repository for the paper Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers.☆30Updated 4 months ago
- This repository contains the pytorch code for our work IEEE ISBI 2024 paper "ConvLoRA and AdaBN Based Domain Adaptation via Self-Training…☆80Updated 9 months ago
- This is the official code for paper: Token Summarisation for Efficient Vision Transformers via Graph-based Token Propagation☆29Updated last year
- An official codebase of paper "Revisiting Sparse Convolutional Model for Visual Recognition"☆124Updated 2 years ago
- code for WaveCNet☆17Updated 4 years ago
- ☆27Updated 2 years ago
- [ICASSP-2021] Official implementations of Multi-View Contrastive Learning for Online Knowledge Distillation (MCL-OKD)☆27Updated 4 years ago
- Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approach, CVPR 2024☆22Updated 11 months ago