Tebmer / Awesome-Knowledge-Distillation-of-LLMsView external linksLinks
This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.
☆1,252Mar 9, 2025Updated 11 months ago
Alternatives and similar repositories for Awesome-Knowledge-Distillation-of-LLMs
Users that are interested in Awesome-Knowledge-Distillation-of-LLMs are comparing it to the libraries listed below
Sorting:
- Official PyTorch implementation of DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024)☆251Mar 13, 2025Updated 11 months ago
- Best practices for distilling large language models.☆604Feb 1, 2024Updated 2 years ago
- A curated list for Efficient Large Language Models☆1,951Jun 17, 2025Updated 8 months ago
- An Open Source Toolkit For LLM Distillation☆860Dec 21, 2025Updated last month
- General technology for enabling AI capabilities w/ LLMs and MLLMs☆4,284Dec 22, 2025Updated last month
- Repo for the EMNLP'24 Paper "Dual-Space Knowledge Distillation for Large Language Models". A general white-box KD framework for both same…☆61Aug 26, 2025Updated 5 months ago
- Awesome LLM compression research papers and tools.☆1,776Nov 10, 2025Updated 3 months ago
- A framework for few-shot evaluation of language models.☆11,393Updated this week
- A pipeline for LLM knowledge distillation☆112Apr 2, 2025Updated 10 months ago
- ☆580Sep 7, 2023Updated 2 years ago
- A family of compressed models obtained via pruning and knowledge distillation☆366Nov 6, 2025Updated 3 months ago
- An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & TIS & vLLM & Ray & Async RL)☆8,989Feb 6, 2026Updated last week
- From Chain-of-Thought prompting to OpenAI o1 and DeepSeek-R1 🍓☆3,538May 7, 2025Updated 9 months ago
- [NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baich…☆1,106Oct 7, 2024Updated last year
- [ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning☆640Mar 4, 2024Updated last year
- O1 Replication Journey☆1,999Jan 14, 2025Updated last year
- verl: Volcano Engine Reinforcement Learning for LLMs☆19,246Updated this week
- Less is More: Task-aware Layer-wise Distillation for Language Model Compression (ICML2023)☆40Aug 28, 2023Updated 2 years ago
- A series of technical report on Slow Thinking with LLM☆759Aug 13, 2025Updated 6 months ago
- Must-read Papers on Knowledge Editing for Large Language Models.☆1,212Jul 12, 2025Updated 7 months ago
- Latest Advances on Multimodal Large Language Models☆17,337Feb 7, 2026Updated last week
- AllenAI's post-training codebase☆3,573Updated this week
- Simple RL training for reasoning☆3,827Dec 23, 2025Updated last month
- Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)☆67,253Updated this week
- 📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥☆1,910Jan 22, 2026Updated 3 weeks ago
- A collection of LLM papers, blogs, and projects, with a focus on OpenAI o1 🍓 and reasoning techniques.☆6,889Dec 17, 2025Updated 2 months ago
- Train transformer language models with reinforcement learning.☆17,360Updated this week
- Fully open reproduction of DeepSeek-R1☆25,879Nov 24, 2025Updated 2 months ago
- 🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.☆20,619Feb 9, 2026Updated last week
- Awesome LLM pruning papers all-in-one repository with integrating all useful resources and insights.☆148Aug 8, 2025Updated 6 months ago
- Awesome list for LLM pruning.☆283Oct 11, 2025Updated 4 months ago
- [ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning☆366Sep 6, 2024Updated last year
- FuseAI Project☆588Jan 25, 2025Updated last year
- Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。☆2,654May 30, 2023Updated 2 years ago
- Robust recipes to align language models with human and AI preferences☆5,495Sep 8, 2025Updated 5 months ago
- [TMLR 2024] Efficient Large Language Models: A Survey☆1,253Jun 23, 2025Updated 7 months ago
- ⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)☆1,003Dec 6, 2024Updated last year
- Reading list of hallucination in LLMs. Check out our new survey paper: "Siren’s Song in the AI Ocean: A Survey on Hallucination in Large …☆1,076Sep 27, 2025Updated 4 months ago
- OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models☆1,830Jan 17, 2025Updated last year