Alternatives and similar repositories for EigenAttn

Users that are interested in EigenAttn are comparing it to the libraries listed below

• ArminAzizi98 / LaMDA
  ☆15Updated 9 months ago
• SempraETY / Pruning-via-Merging
  ☆20Updated 8 months ago
• Lucky-Lance / SPP
  [ICML 2024] SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
  ☆21Updated last year
• AkideLiu / MiniCache
  ☆10Updated 11 months ago
• TaiMingLu / know-dont-tell
  ☆16Updated 9 months ago
• r-three / smear
  ☆30Updated last year
• dongwonjo / FastKV
  Official Implementation of FastKV: KV Cache Compression for Fast Long-Context Processing with Token-Selective Propagation
  ☆22Updated 2 months ago
• VijayLingam95 / SVFT
  ☆30Updated 6 months ago
• yule-BUAA / MergeLLM
  Codes for Merging Large Language Models
  ☆33Updated last year
• UCSB-NLP-Chang / ThinkPrune
  ☆39Updated 3 months ago
• jiwonsong-dev / SLEB
  Official Implementation of SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
  ☆38Updated 6 months ago
• gccnlp / Light-PEFT
  [ACL 2024 Findings] Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning
  ☆11Updated 11 months ago
• ROIM1998 / APT
  [ICML'24 Oral] APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
  ☆44Updated last year
• peijunallin / alphalora
  ☆18Updated 9 months ago
• declare-lab / della
  DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling
  ☆34Updated last year
• BaohaoLiao / ApiQ
  [EMNLP 2024] Quantize LLM to extremely low-bit, and finetune the quantized LLMs
  ☆13Updated last year
• shoaibahmed / llm_depth_pruning
  Official implementation of the paper: "A deeper look at depth pruning of LLMs"
  ☆15Updated last year
• FeiyuZhang98 / IncreLoRA
  ☆34Updated last year
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆28Updated last year
• ShiZhengyan / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆38Updated last year
• pixeli99 / MixLN
  [ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxia…
  ☆25Updated 2 weeks ago
• GATECH-EIC / ACT
  [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…
  ☆40Updated last year
• Lucky-Lance / Expert_Sparsity
  [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  ☆97Updated last year
• OpenSparseLLMs / Linearization
  ☆54Updated last month
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆52Updated 6 months ago
• Infini-AI-Lab / S2FT
  ☆19Updated 7 months ago
• hemingkx / SWIFT
  [ICLR 2025] SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
  ☆52Updated 5 months ago
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆61Updated 10 months ago
• Anonymous1252022 / Megatron-DeepSpeed
  ☆12Updated 10 months ago
• pprp / Pruner-Zero
  [ICML24] Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆91Updated 8 months ago