songmzhang / DSKD
Repo for the EMNLP'24 Paper "Dual-Space Knowledge Distillation for Large Language Models". A general white-box KD framework for both same-tokenizer and cross-tokenizer LLM distillation.
☆43Updated 4 months ago
Alternatives and similar repositories for DSKD:
Users that are interested in DSKD are comparing it to the libraries listed below
- TokenSkip: Controllable Chain-of-Thought Compression in LLMs☆98Updated last week
- [ICLR 2025] 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training (Spotlight)☆115Updated last month
- L-CITEEVAL: DO LONG-CONTEXT MODELS TRULY LEVERAGE CONTEXT FOR RESPONDING?☆23Updated 5 months ago
- [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".☆116Updated 4 months ago
- [ICML 2024] Selecting High-Quality Data for Training Language Models☆158Updated 9 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆76Updated last year
- [EMNLP 2024] Source code for the paper "Learning Planning-based Reasoning with Trajectory Collection and Process Rewards Synthesizing".☆73Updated 2 months ago
- [ICML'2024] Can AI Assistants Know What They Don't Know?☆79Updated last year
- ☆98Updated 5 months ago
- [NAACL 2024 Outstanding Paper] Source code for the NAACL 2024 paper entitled "R-Tuning: Instructing Large Language Models to Say 'I Don't…☆109Updated 8 months ago
- ☆48Updated last month
- ☆127Updated 8 months ago
- ☆17Updated 3 months ago
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆147Updated 6 months ago
- We introduce ScaleQuest, a scalable, novel and cost-effective data synthesis method to unleash the reasoning capability of LLMs.☆60Updated 4 months ago
- ☆80Updated last year
- LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models☆73Updated 5 months ago
- Official repository for paper "Weak-to-Strong Extrapolation Expedites Alignment"☆73Updated 9 months ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆152Updated 9 months ago
- [ACL 2024 (Oral)] A Prospector of Long-Dependency Data for Large Language Models☆54Updated 8 months ago
- ☆27Updated last month
- One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning☆39Updated last year
- [SIGIR'24] The official implementation code of MOELoRA.☆153Updated 8 months ago
- [ICLR'25] Data and code for our paper "Why Does the Effective Context Length of LLMs Fall Short?"☆70Updated 3 months ago
- Code associated with Tuning Language Models by Proxy (Liu et al., 2024)☆107Updated 11 months ago
- [ACL 2024] Learning to Edit: Aligning LLMs with Knowledge Editing☆35Updated 7 months ago
- Code & Data for our Paper "Alleviating Hallucinations of Large Language Models through Induced Hallucinations"☆63Updated last year
- A method of ensemble learning for heterogeneous large language models.☆41Updated 7 months ago
- ☆59Updated 6 months ago
- ☆31Updated last year