Alternatives and similar repositories for Knowledge-Distillation

Users that are interested in Knowledge-Distillation are comparing it to the libraries listed below

• kytimmylai / NoisyNN-PyTorch
  non-official NoisyNN Implemnentation
  ☆50Updated last year
• hkproj / mamba-notes
  Notes on the Mamba and the S4 model (Mamba: Linear-Time Sequence Modeling with Selective State Spaces)
  ☆169Updated last year
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆209Updated last year
• Jaykef / ai-algorithms
  First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…
  ☆178Updated 3 weeks ago
• hkproj / pytorch-transformer-distributed
  Distributed training (multi-node) of a Transformer model
  ☆76Updated last year
• jacobmarks / awesome-neurips-2023
  Conference schedule, top papers, and analysis of the data for NeurIPS 2023!
  ☆119Updated last year
• kyegomez / MambaTransformer
  Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling
  ☆202Updated this week
• kyegomez / Jamba
  PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"
  ☆182Updated last week
• rasbt / cvpr2023
  ☆133Updated last year
• kyegomez / xLSTM
  Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"
  ☆119Updated 3 weeks ago
• hkproj / pytorch-lora
  LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch
  ☆112Updated 2 years ago
• mishra-18 / ML-Models
  ☆43Updated last month
• AviSoori1x / seemore
  From scratch implementation of a vision language model in pure PyTorch
  ☆234Updated last year
• fkodom / yet-another-retnet
  A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…
  ☆105Updated last year
• lucidrains / pytorch-custom-utils
  Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…
  ☆124Updated last year
• hkproj / pytorch-ddpm
  Implementation of the paper "Denoising Diffusion Probabilistic Models" in PyTorch
  ☆64Updated 2 years ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆111Updated 7 months ago
• myscience / x-lstm
  Pytorch implementation of the xLSTM model by Beck et al. (2024)
  ☆169Updated last year
• shreydan / VisionGPT2
  Combining ViT and GPT-2 for image captioning. Trained on MS-COCO. The model was implemented mostly from scratch.
  ☆43Updated last year
• Weixin-Liang / Mixture-of-Mamba
  ☆47Updated 6 months ago
• lucidrains / soft-moe-pytorch
  Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch
  ☆311Updated 4 months ago
• ByungKwanLee / DeepSick-R1
  Reproduction of DeepSeek-R1
  ☆237Updated 4 months ago
• fangyuan-ksgk / Mini-LLaVA
  A minimal implementation of LLaVA-style VLM with interleaved image & text & video processing ability.
  ☆95Updated 7 months ago
• kyleliang919 / C-Optim
  When it comes to optimizers, it's always better to be safe than sorry
  ☆356Updated this week
• akaashdash / kansformers
  ☆136Updated last year
• Hon-Wong / VoRA
  [Fully open] [Encoder-free MLLM] Vision as LoRA
  ☆326Updated 2 months ago
• apapiu / mamba_small_bench
  Trying out the Mamba architecture on small examples (cifar-10, shakespeare char level etc.)
  ☆48Updated last year
• WalBouss / LeGrad
  [ICCV25] Official Implementation of LeGrad
  ☆78Updated 10 months ago
• tintn / vision-transformer-from-scratch
  A Simplified PyTorch Implementation of Vision Transformer (ViT)
  ☆197Updated last year
• a-r-r-o-w / kanformer
  Naively combining transformers and Kolmogorov-Arnold Networks to learn and experiment
  ☆36Updated last year