Alternatives and similar repositories for Fira:

Users that are interested in Fira are comparing it to the libraries listed below

• OpenSparseLLMs / Linear-MoE
  ☆72Updated this week
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆75Updated 9 months ago
• hemingkx / TokenSkip
  TokenSkip: Controllable Chain-of-Thought Compression in LLMs
  ☆103Updated 2 weeks ago
• YouAreSpecialToMe / QST
  Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
  ☆42Updated 4 months ago
• horseee / CoT-Valve
  CoT-Valve: Length-Compressible Chain-of-Thought Tuning
  ☆56Updated last month
• sail-sg / LongSpec
  LongSpec: Long-Context Speculative Decoding with Efficient Drafting and Verification
  ☆42Updated 3 weeks ago
• mit-han-lab / x-attention
  XAttention: Block Sparse Attention with Antidiagonal Scoring
  ☆118Updated this week
• OpenSparseLLMs / MoM
  ☆73Updated last week
• sail-sg / SimLayerKV
  The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.
  ☆44Updated 5 months ago
• TUDB-Labs / MixLoRA
  State-of-the-art Parameter-Efficient MoE Fine-tuning Method
  ☆152Updated 7 months ago
• Lucky-Lance / Expert_Sparsity
  [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  ☆86Updated 10 months ago
• thu-nics / MoA
  The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>
  ☆122Updated 3 months ago
• Chongjie-Si / Subspace-Tuning
  A generalized framework for subspace tuning methods in parameter efficient fine-tuning.
  ☆131Updated last month
• step-law / steplaw
  ☆143Updated 2 weeks ago
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆63Updated 11 months ago
• thu-ml / ReMoE
  [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.
  ☆65Updated 3 months ago
• OpenSparseLLMs / Linearization
  ☆37Updated last week
• GAIR-NLP / LIMR
  ☆171Updated last month
• CASE-Lab-UMD / Unified-MoE-Compression
  The official implementation of the paper "Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques (TMLR)".
  ☆62Updated last week
• cmu-l3 / l1
  L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
  ☆162Updated last week
• hyx1999 / SAM-Decoding
  Official Implementation of SAM-Decoding: Speculative Decoding via Suffix Automaton
  ☆23Updated last month
• hahnyuan / ASVD4LLM
  Activation-aware Singular Value Decomposition for Compressing Large Language Models
  ☆59Updated 5 months ago
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆154Updated 9 months ago
• wutaiqiang / MoSLoRA
  ☆99Updated 8 months ago
• pprp / Pruner-Zero
  Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆80Updated 4 months ago
• mutonix / pyramidinfer
  ☆39Updated 4 months ago
• GCYZSL / MoLA
  ☆131Updated 8 months ago
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆39Updated last year
• LINs-lab / DynMoE
  [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
  ☆80Updated last month
• YangLing0818 / SuperCorrect-llm
  [ICLR 2025] SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction
  ☆64Updated last week