Alternatives and similar repositories for llm-distillation

Users that are interested in llm-distillation are comparing it to the libraries listed below

• Nicolas-BZRD / llm-recipes
  ☆29Updated last year
• shankarp8 / knowledge_distillation
  Repository for "Propagating Knowledge Updates to LMs Through Distillation" (NeurIPS 2023).
  ☆25Updated 10 months ago
• yuzhaouoe / pretraining-data-packing
  [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training
  ☆22Updated 10 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆52Updated 2 years ago
• prateeky2806 / ComPEFT
  ☆25Updated last year
• r-three / RAD
  Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model
  ☆43Updated last year
• swj0419 / in-context-pretraining
  ☆51Updated last year
• allenai / hyperdecoders
  Codebase for Hyperdecoders https://arxiv.org/abs/2203.08304
  ☆11Updated 2 years ago
• IST-DASLab / RoSA
  Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)
  ☆42Updated last year
• allenai / hyper-task-descriptions
  Learning adapter weights from task descriptions
  ☆19Updated last year
• jzbjyb / ReAtt
  Retrieval as Attention
  ☆82Updated 2 years ago
• nayeon7lee / FactualityPrompt
  ☆86Updated 2 years ago
• ekinakyurek / influence
  Code for "Tracing Knowledge in Language Models Back to the Training Data"
  ☆38Updated 2 years ago
• allenai / bff
  ☆38Updated last year
• anadim / the-little-retrieval-test
  ☆34Updated 2 years ago
• GSYfate / knnlm-limits
  Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"
  ☆23Updated last month
• hamishivi / automated-instruction-selection
  Exploration of automated dataset selection approaches at large scales.
  ☆45Updated 3 months ago
• Nanami18 / Snowballed_Hallucination
  ☆44Updated 9 months ago
• Zayne-sprague / MuSR
  ☆44Updated 10 months ago
• ThomasScialom / T0_continual_learning
  Adding new tasks to T0 without catastrophic forgetting
  ☆33Updated 2 years ago
• Raibows / Learn-to-Reason
  Code for "Democratizing Reasoning Ability: Tailored Learning from Large Language Model", EMNLP 2023
  ☆35Updated last year
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆57Updated 9 months ago
• amazon-science / ContraCLM
  [ACL 2023] Code for ContraCLM: Contrastive Learning For Causal Language Model
  ☆34Updated last year
• yasumasaonoe / entity_knowledge_propagation
  ☆17Updated last year
• csarron / BTR
  ☆16Updated last year
• TristanThrush / perplexity-correlations
  Simple and scalable tools for data-driven pretraining data selection.
  ☆24Updated 2 weeks ago
• ShiZhengyan / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆38Updated last year
• hadasah / btm
  ☆74Updated last year
• kaistAI / Janus
  [NeurIPS 2024] Train LLMs with diverse system messages reflecting individualized preferences to generalize to unseen system messages
  ☆48Updated 6 months ago
• frankxu2004 / knnlm-why
  Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"
  ☆57Updated 2 years ago