Alternatives and similar repositories for Knowledge-Distillation

Users that are interested in Knowledge-Distillation are comparing it to the libraries listed below

• AviSoori1x / seemore
  From scratch implementation of a vision language model in pure PyTorch
  ☆252Updated last year
• rasbt / cvpr2023
  ☆134Updated 2 years ago
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆215Updated last year
• hkproj / pytorch-transformer-distributed
  Distributed training (multi-node) of a Transformer model
  ☆89Updated last year
• shreydan / VisionGPT2
  Combining ViT and GPT-2 for image captioning. Trained on MS-COCO. The model was implemented mostly from scratch.
  ☆46Updated 2 years ago
• kyegomez / Jamba
  PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"
  ☆200Updated last month
• hkproj / mamba-notes
  Notes on the Mamba and the S4 model (Mamba: Linear-Time Sequence Modeling with Selective State Spaces)
  ☆175Updated last year
• myscience / x-lstm
  Pytorch implementation of the xLSTM model by Beck et al. (2024)
  ☆180Updated last year
• WalBouss / LeGrad
  [ICCV25] Official Implementation of LeGrad
  ☆83Updated last year
• NielsRogge / Vision-Transformer-papers
  This repository contains an overview of important follow-up works based on the original Vision Transformer (ViT) by Google.
  ☆187Updated 3 years ago
• kyegomez / MambaTransformer
  Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling
  ☆211Updated last month
• jacobmarks / awesome-neurips-2023
  Conference schedule, top papers, and analysis of the data for NeurIPS 2023!
  ☆121Updated 2 years ago
• lucidrains / pytorch-custom-utils
  Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…
  ☆125Updated last year
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆113Updated 11 months ago
• hkproj / pytorch-lora
  LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch
  ☆117Updated 2 years ago
• fkodom / yet-another-retnet
  A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…
  ☆106Updated 2 years ago
• catid / dora
  Implementation of DoRA
  ☆307Updated last year
• Weixin-Liang / Mixture-of-Mamba
  ☆50Updated 10 months ago
• ariG23498 / quantized-diffusion-inference
  Notebook and Scripts that showcase running quantized diffusion models on consumer GPUs
  ☆38Updated last year
• myscience / mamba
  Pytorch (Lightning) implementation of the Mamba model
  ☆33Updated 7 months ago
• hkproj / quantization-notes
  Notes on quantization in neural networks
  ☆113Updated 2 years ago
• hkproj / pytorch-ddpm
  Implementation of the paper "Denoising Diffusion Probabilistic Models" in PyTorch
  ☆67Updated 2 years ago
• PeaBrane / mamba-tiny
  Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).
  ☆128Updated last year
• knotgrass / attention
  several types of attention modules written in PyTorch for learning purposes
  ☆52Updated last year
• apapiu / mamba_small_bench
  Trying out the Mamba architecture on small examples (cifar-10, shakespeare char level etc.)
  ☆47Updated 2 years ago
• tintn / vision-transformer-from-scratch
  A Simplified PyTorch Implementation of Vision Transformer (ViT)
  ☆224Updated last year
• kytimmylai / NoisyNN-PyTorch
  non-official NoisyNN Implemnentation
  ☆50Updated last year
• hkproj / multi-latent-attention
  ☆45Updated 6 months ago
• andimarafioti / florence2-finetuning
  Quick exploration into fine tuning florence 2
  ☆336Updated last year
• ariG23498 / fine-tune-paligemma
  Notebooks for fine tuning pali gemma
  ☆117Updated 8 months ago