josehoras / Knowledge-DistillationLinks
☆11Updated 5 years ago
Alternatives and similar repositories for Knowledge-Distillation
Users that are interested in Knowledge-Distillation are comparing it to the libraries listed below
Sorting:
- LoRA and DoRA from Scratch Implementations☆214Updated last year
- From scratch implementation of a vision language model in pure PyTorch☆250Updated last year
- Notes on the Mamba and the S4 model (Mamba: Linear-Time Sequence Modeling with Selective State Spaces)☆173Updated last year
- Notes on quantization in neural networks☆105Updated last year
- LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch☆117Updated 2 years ago
- Conference schedule, top papers, and analysis of the data for NeurIPS 2023!☆121Updated last year
- Combining ViT and GPT-2 for image captioning. Trained on MS-COCO. The model was implemented mostly from scratch.☆46Updated 2 years ago
- PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"☆195Updated 3 weeks ago
- ☆134Updated 2 years ago
- Implementation of DoRA☆307Updated last year
- Distributed training (multi-node) of a Transformer model☆86Updated last year
- several types of attention modules written in PyTorch for learning purposes☆52Updated last year
- First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…☆179Updated 4 months ago
- Research projects built on top of Transformers☆100Updated 8 months ago
- Prune transformer layers☆74Updated last year
- Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).☆127Updated last year
- Complete implementation of Llama2 with/without KV cache & inference 🚀☆48Updated last year
- Documentation, notes, links, etc for streams.☆84Updated last year
- A Simplified PyTorch Implementation of Vision Transformer (ViT)☆220Updated last year
- Implementation of Infini-Transformer in Pytorch☆113Updated 10 months ago
- A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…☆106Updated last year
- Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling☆210Updated last month
- A easy, reliable, fluid template for python packages complete with docs, testing suites, readme's, github workflows, linting and much muc…☆194Updated last month
- Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…☆123Updated last year
- [ICCV25] Official Implementation of LeGrad☆82Updated last year
- Implementation of Agent Attention in Pytorch☆92Updated last year
- Notebook and Scripts that showcase running quantized diffusion models on consumer GPUs☆38Updated last year
- A minimal implementation of LLaVA-style VLM with interleaved image & text & video processing ability.☆96Updated 11 months ago
- ☆45Updated 5 months ago
- (Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from …☆182Updated last year