Alternatives and similar repositories for Knowledge-Distillation

Users that are interested in Knowledge-Distillation are comparing it to the libraries listed below

• AviSoori1x / seemore
  From scratch implementation of a vision language model in pure PyTorch
  ☆254Updated last year
• hkproj / mamba-notes
  Notes on the Mamba and the S4 model (Mamba: Linear-Time Sequence Modeling with Selective State Spaces)
  ☆179Updated 2 years ago
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆215Updated last year
• hkproj / pytorch-transformer-distributed
  Distributed training (multi-node) of a Transformer model
  ☆93Updated last year
• jacobmarks / awesome-neurips-2023
  Conference schedule, top papers, and analysis of the data for NeurIPS 2023!
  ☆120Updated 2 years ago
• hkproj / pytorch-lora
  LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch
  ☆122Updated 2 years ago
• Jaykef / ai-algorithms
  First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…
  ☆181Updated 6 months ago
• rasbt / cvpr2023
  ☆132Updated 2 years ago
• hkproj / multi-latent-attention
  ☆46Updated 8 months ago
• shreydan / VisionGPT2
  Combining ViT and GPT-2 for image captioning. Trained on MS-COCO. The model was implemented mostly from scratch.
  ☆48Updated 2 years ago
• knotgrass / attention
  several types of attention modules written in PyTorch for learning purposes
  ☆53Updated last month
• hkproj / pytorch-ddpm
  Implementation of the paper "Denoising Diffusion Probabilistic Models" in PyTorch
  ☆67Updated 2 years ago
• WalBouss / LeGrad
  [ICCV25] Official Implementation of LeGrad
  ☆87Updated last year
• kyegomez / Jamba
  PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"
  ☆206Updated 3 weeks ago
• myscience / x-lstm
  Pytorch implementation of the xLSTM model by Beck et al. (2024)
  ☆181Updated last year
• apple / ml-sigmoid-attention
  ☆307Updated 9 months ago
• NielsRogge / Vision-Transformer-papers
  This repository contains an overview of important follow-up works based on the original Vision Transformer (ViT) by Google.
  ☆192Updated 4 years ago
• tintn / vision-transformer-from-scratch
  A Simplified PyTorch Implementation of Vision Transformer (ViT)
  ☆235Updated last year
• Weixin-Liang / Mixture-of-Mamba
  ☆50Updated last year
• catid / dora
  Implementation of DoRA
  ☆306Updated last year
• PeaBrane / mamba-tiny
  Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).
  ☆130Updated last year
• mishra-18 / ML-Models
  ☆48Updated 7 months ago
• Montinger / Transformer-Workbench
  Playground for Transformers
  ☆53Updated 2 years ago
• kyegomez / MambaTransformer
  Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling
  ☆213Updated last week
• fangyuan-ksgk / Mini-LLaVA
  A minimal implementation of LLaVA-style VLM with interleaved image & text & video processing ability.
  ☆98Updated last year
• ThinamXx / Meta-llama
  Complete implementation of Llama2 with/without KV cache & inference 🚀
  ☆49Updated last year
• wjbmattingly / qwen2-vl-finetune-huggingface
  This project is a collection of fine-tuning scripts to help researchers fine-tune Qwen 2 VL on HuggingFace datasets.
  ☆77Updated 6 months ago
• adensur / blog
  ☆75Updated 9 months ago
• fkodom / yet-another-retnet
  A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…
  ☆106Updated 2 years ago
• MrPio / KAN-Continual_Learning_tests
  Collection of tests performed during the study of the new Kolmogorov-Arnold Neural Networks (KAN)
  ☆41Updated 11 months ago