nbasyl / DoRA

Related projects ⓘ

Alternatives and complementary repositories for DoRA

• jongwooko / distillm
  Official PyTorch implementation of DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024)
  ☆139Updated 2 months ago
• jxiw / MambaInLlama
  [NeurIPS 2024] Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models
  ☆174Updated this week
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆119Updated last year
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆64Updated 5 months ago
• GraphPKU / PiSSA
  PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024 Spotlight)
  ☆265Updated this week
• SalesforceAIResearch / GemFilter
  ☆64Updated last month
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆28Updated 7 months ago
• locuslab / massive-activations
  Code accompanying the paper "Massive Activations in Large Language Models"
  ☆123Updated 8 months ago
• yxli2123 / LoftQ
  ☆199Updated 5 months ago
• HanGuo97 / lq-lora
  ☆122Updated 10 months ago
• Outsider565 / LoRA-GA
  ☆155Updated last month
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (Official Code)
  ☆135Updated last month
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆135Updated 5 months ago
• wuhy68 / Parameter-Efficient-MoE
  Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
  ☆129Updated 2 months ago
• hahnyuan / ASVD4LLM
  Activation-aware Singular Value Decomposition for Compressing Large Language Models
  ☆49Updated last month
• 42Shawn / LLaVA-PruMerge
  LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models
  ☆100Updated 6 months ago
• snu-mllab / Context-Memory
  Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
  ☆50Updated 7 months ago
• mrflogs / LoRA-Pro
  Official code for our paper, "LoRA-Pro: Are Low-Rank Adapters Properly Optimized? "
  ☆74Updated last month
• nikhil-ghosh-berkeley / loraplus
  ☆200Updated 4 months ago
• YuchuanTian / DiJiang
  [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…
  ☆98Updated 5 months ago
• haonan3 / AnchorContext
  AnchorAttention: Improved attention for LLMs long-context training
  ☆142Updated this week
• thu-nics / MoA
  The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>
  ☆101Updated last week
• FasterDecoding / TEAL
  ☆96Updated 2 months ago
• NVlabs / MaskLLM
  [NeurIPS 24 Spotlight] MaskLLM: Learnable Semi-structured Sparsity for Large Language Models
  ☆120Updated 3 weeks ago
• GCYZSL / MoLA
  ☆116Updated 4 months ago
• GATECH-EIC / ACT
  [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…
  ☆24Updated 4 months ago
• CASE-Lab-UMD / LLM-Drop
  The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".
  ☆143Updated last week
• prateeky2806 / ties-merging
  ☆153Updated 9 months ago
• FreedomIntelligence / MLLM-Bench
  MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
  ☆55Updated last month
• microsoft / rho
  Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.
  ☆307Updated 7 months ago