Alternatives and similar repositories for KaSA:

Users that are interested in KaSA are comparing it to the libraries listed below

• uservan / ThinkPO
  ☆17Updated last month
• mathllm / Step-Controlled_DPO
  ☆21Updated 8 months ago
• joonkeekim / Instructive-Decoding
  Official repository of "Distort, Distract, Decode: Instruction-Tuned Model Can Refine its Response from Noisy Instructions", ICLR 2024 Sp…
  ☆20Updated last year
• ZhengxiangShi / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆35Updated 10 months ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆33Updated 6 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆48Updated 2 years ago
• snu-mllab / Context-Memory
  Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
  ☆55Updated 11 months ago
• Infini-AI-Lab / S2FT
  ☆16Updated 2 months ago
• scitix / MEAP
  Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
  ☆27Updated last month
• sail-sg / SkyLadder
  The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling
  ☆23Updated last week
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆46Updated last month
• PKU-ML / LongPPL
  Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"
  ☆44Updated this week
• JayZhang42 / SLED
  SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Model https://arxiv.org/pdf/2411.02433
  ☆24Updated 3 months ago
• yale-nlp / refdpo
  ☆15Updated 8 months ago
• ZetangForward / LCM_Stack
  Code for paper: Long cOntext aliGnment via efficient preference Optimization
  ☆13Updated last month
• FeiyuZhang98 / IncreLoRA
  ☆31Updated last year
• IST-DASLab / RoSA
  Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)
  ☆39Updated last year
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆50Updated 6 months ago
• GeniusHTX / TALE
  ☆79Updated last week
• KindXiaoming / physics_of_skill_learning
  We study toy models of skill learning.
  ☆24Updated 2 months ago
• yule-BUAA / MergeLLM
  Codes for Merging Large Language Models
  ☆28Updated 7 months ago
• linkedin / ControlLLM
  Control LLM
  ☆13Updated this week
• YangLing0818 / SuperCorrect-llm
  [ICLR 2025] SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction
  ☆64Updated last week
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆37Updated 5 months ago
• dmis-lab / Monet
  [ICLR 2025] Monet: Mixture of Monosemantic Experts for Transformers
  ☆60Updated 2 months ago
• DAMO-NLP-SG / LongPO
  [ICLR 2025] LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
  ☆31Updated last month
• OpenSparseLLMs / Linearization
  ☆37Updated last week
• kyegomez / MultiQueryAttention
  This is a simple torch implementation of the high performance Multi-Query Attention
  ☆16Updated last year
• Arenaa / Accelerated-Generation-Techniques
  This repository contains papers for a comprehensive survey on accelerated generation techniques in Large Language Models (LLMs).
  ☆11Updated 10 months ago
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆44Updated 8 months ago