Nicolas-BZRD / llm-distillation

Related projects ⓘ

Alternatives and complementary repositories for llm-distillation

• Nicolas-BZRD / llm-recipes
  ☆24Updated 8 months ago
• shankarp8 / knowledge_distillation
  Repository for "Propagating Knowledge Updates to LMs Through Distillation" (NeurIPS 2023).
  ☆24Updated 3 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆44Updated last year
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆53Updated last month
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆39Updated 2 months ago
• prateeky2806 / ComPEFT
  ☆25Updated last year
• songmzhang / DSKD
  Repo for the EMNLP'24 Paper "Dual-Space Knowledge Distillation for Large Language Models".
  ☆37Updated 2 weeks ago
• IST-DASLab / RoSA
  ☆35Updated 9 months ago
• anadim / the-little-retrieval-test
  ☆31Updated last year
• jzbjyb / ReAtt
  Retrieval as Attention
  ☆83Updated last year
• yuzhaouoe / pretraining-data-packing
  [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training
  ☆18Updated 3 months ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆32Updated 2 months ago
• zjysteven / mink-plus-plus
  Min-K%++: Improved baseline for detecting pre-training data of LLMs https://arxiv.org/abs/2404.02936
  ☆27Updated 5 months ago
• cxcscmu / MATES
  Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]
  ☆49Updated last week
• liyucheng09 / Contamination_Detector
  Lightweight tool to identify Data Contamination in LLMs evaluation
  ☆42Updated 8 months ago
• epfl-dlab / llm-latent-language
  Repo accompanying our paper "Do Llamas Work in English? On the Latent Language of Multilingual Transformers".
  ☆58Updated 8 months ago
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆15Updated 5 months ago
• joeljang / ELM
  [ICML 2023] Exploring the Benefits of Training Expert Language Models over Instruction Tuning
  ☆97Updated last year
• fc2869 / lo-fit
  LoFiT: Localized Fine-tuning on LLM Representations
  ☆21Updated 5 months ago
• kamanphoebe / Look-into-MoEs
  A Closer Look into Mixture-of-Experts in Large Language Models
  ☆40Updated 3 months ago
• GSYfate / knnlm-limits
  Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"
  ☆18Updated 2 months ago
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆28Updated 7 months ago
• Zayne-sprague / MuSR
  ☆34Updated 3 months ago
• r-three / smear
  ☆27Updated last year
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆71Updated last month
• pratyushmaini / llm_dataset_inference
  Official Repository for Dataset Inference for LLMs
  ☆23Updated 4 months ago
• activatedgeek / calibration-tuning
  ☆36Updated 3 months ago
• frankxu2004 / knnlm-why
  Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"
  ☆56Updated last year
• TIGER-AI-Lab / LongICLBench
  Code and Data for "Long-context LLMs Struggle with Long In-context Learning"
  ☆91Updated 4 months ago
• jongwooko / distillm
  Official PyTorch implementation of DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024)
  ☆139Updated 2 months ago