Alternatives and similar repositories for Knowledge-Distillation

Users that are interested in Knowledge-Distillation are comparing it to the libraries listed below

• hkproj / pytorch-lora
  LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch
  ☆122Updated 2 years ago
• hkproj / mamba-notes
  Notes on the Mamba and the S4 model (Mamba: Linear-Time Sequence Modeling with Selective State Spaces)
  ☆179Updated 2 years ago
• rasbt / cvpr2023
  ☆132Updated 2 years ago
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆215Updated last year
• shreydan / VisionGPT2
  Combining ViT and GPT-2 for image captioning. Trained on MS-COCO. The model was implemented mostly from scratch.
  ☆48Updated 2 years ago
• hkproj / pytorch-transformer-distributed
  Distributed training (multi-node) of a Transformer model
  ☆93Updated last year
• AviSoori1x / seemore
  From scratch implementation of a vision language model in pure PyTorch
  ☆254Updated last year
• jacobmarks / awesome-neurips-2023
  Conference schedule, top papers, and analysis of the data for NeurIPS 2023!
  ☆120Updated 2 years ago
• hkproj / quantization-notes
  Notes on quantization in neural networks
  ☆117Updated 2 years ago
• kytimmylai / NoisyNN-PyTorch
  non-official NoisyNN Implemnentation
  ☆50Updated 2 years ago
• lucidrains / contrastive-rl
  Contrastive Reinforcement Learning
  ☆59Updated last week
• PeaBrane / mamba-tiny
  Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).
  ☆130Updated last year
• tintn / vision-transformer-from-scratch
  A Simplified PyTorch Implementation of Vision Transformer (ViT)
  ☆235Updated last year
• kyegomez / MambaTransformer
  Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling
  ☆213Updated last week
• kyegomez / Jamba
  PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"
  ☆206Updated 3 weeks ago
• ThinamXx / Meta-llama
  Complete implementation of Llama2 with/without KV cache & inference 🚀
  ☆49Updated last year
• lucidrains / soft-moe-pytorch
  Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch
  ☆344Updated 10 months ago
• Jaykef / ai-algorithms
  First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…
  ☆181Updated 6 months ago
• lucidrains / pytorch-custom-utils
  Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…
  ☆126Updated last year
• lucidrains / st-moe-pytorch
  Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
  ☆378Updated last year
• lucidrains / agent-attention-pytorch
  Implementation of Agent Attention in Pytorch
  ☆93Updated last year
• hkproj / multi-latent-attention
  ☆46Updated 8 months ago
• kyegomez / zeta
  Build high-performance AI models with modular building blocks
  ☆576Updated last week
• SkalskiP / awesome-foundation-and-multimodal-models
  👁️ + 💬 + 🎧 = 🤖 Curated list of top foundation and multimodal models! [Paper + Code + Examples + Tutorials]
  ☆637Updated last year
• knotgrass / attention
  several types of attention modules written in PyTorch for learning purposes
  ☆53Updated last month
• kyegomez / Python-Package-Template
  A easy, reliable, fluid template for python packages complete with docs, testing suites, readme's, github workflows, linting and much muc…
  ☆199Updated last week
• catid / dora
  Implementation of DoRA
  ☆306Updated last year
• TheSeriousProgrammer / SimpleBitNet
  Simple Adaptation of BitNet
  ☆32Updated last year
• Animadversio / DiffusionFromScratch
  Rebuild the Stable Diffusion Model in a single python script. Tutorial for Harvard ML from Scratch Series
  ☆224Updated last year
• melisa-writer / short-transformers
  Prune transformer layers
  ☆74Updated last year