Alternatives and similar repositories for Knowledge-Distillation

Users that are interested in Knowledge-Distillation are comparing it to the libraries listed below

• AviSoori1x / seemore
  From scratch implementation of a vision language model in pure PyTorch
  ☆246Updated last year
• hkproj / mamba-notes
  Notes on the Mamba and the S4 model (Mamba: Linear-Time Sequence Modeling with Selective State Spaces)
  ☆173Updated last year
• kyegomez / Jamba
  PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"
  ☆192Updated last week
• hkproj / pytorch-lora
  LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch
  ☆117Updated 2 years ago
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆211Updated last year
• Jaykef / ai-algorithms
  First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…
  ☆178Updated 3 months ago
• kyegomez / zeta
  Build high-performance AI models with modular building blocks
  ☆559Updated last week
• aykutcayir34 / DifferentialTransformer
  ☆13Updated last year
• knotgrass / attention
  several types of attention modules written in PyTorch for learning purposes
  ☆52Updated last year
• shreydan / VisionGPT2
  Combining ViT and GPT-2 for image captioning. Trained on MS-COCO. The model was implemented mostly from scratch.
  ☆44Updated 2 years ago
• rasbt / cvpr2023
  ☆134Updated 2 years ago
• kyegomez / MambaTransformer
  Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling
  ☆207Updated 2 weeks ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆113Updated 9 months ago
• myscience / x-lstm
  Pytorch implementation of the xLSTM model by Beck et al. (2024)
  ☆176Updated last year
• hkproj / quantization-notes
  Notes on quantization in neural networks
  ☆104Updated last year
• myscience / mamba
  Pytorch (Lightning) implementation of the Mamba model
  ☆31Updated 6 months ago
• kyegomez / xLSTM
  Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"
  ☆118Updated last week
• PeaBrane / mamba-tiny
  Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).
  ☆125Updated last year
• a-r-r-o-w / kanformer
  Naively combining transformers and Kolmogorov-Arnold Networks to learn and experiment
  ☆36Updated last year
• jacobmarks / awesome-neurips-2023
  Conference schedule, top papers, and analysis of the data for NeurIPS 2023!
  ☆121Updated last year
• lucidrains / pytorch-custom-utils
  Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…
  ☆123Updated last year
• angry-kratos / Simple_Llama3_from_scratch
  ☆30Updated last year
• apple / ml-sigmoid-attention
  ☆302Updated 6 months ago
• tintn / vision-transformer-from-scratch
  A Simplified PyTorch Implementation of Vision Transformer (ViT)
  ☆218Updated last year
• apapiu / mamba_small_bench
  Trying out the Mamba architecture on small examples (cifar-10, shakespeare char level etc.)
  ☆47Updated last year
• NielsRogge / Vision-Transformer-papers
  This repository contains an overview of important follow-up works based on the original Vision Transformer (ViT) by Google.
  ☆185Updated 3 years ago
• kytimmylai / NoisyNN-PyTorch
  non-official NoisyNN Implemnentation
  ☆50Updated last year
• kyleliang919 / C-Optim
  When it comes to optimizers, it's always better to be safe than sorry
  ☆376Updated last month
• hkproj / multi-latent-attention
  ☆45Updated 5 months ago
• andrewgcodes / xlstm
  my attempts at implementing various bits of Sepp Hochreiter's new xLSTM architecture
  ☆131Updated last year