Alternatives and similar repositories for loretta:

Users that are interested in loretta are comparing it to the libraries listed below

• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆60Updated 9 months ago
• yxli2123 / LoSparse
  ☆49Updated last year
• ArminAzizi98 / LaMDA
  ☆13Updated 2 months ago
• zyxxmu / cam
  Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference
  ☆29Updated 7 months ago
• sanagno / adaptively_sparse_attention
  ☆18Updated last year
• Shwai-He / MEO
  The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":
  ☆34Updated 9 months ago
• YouAreSpecialToMe / QST
  Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
  ☆42Updated 2 months ago
• TsinghuaC3I / SoRA
  [EMNLP 2023 Main] Sparse Low-rank Adaptation of Pre-trained Language Models
  ☆70Updated 10 months ago
• SempraETY / Pruning-via-Merging
  ☆16Updated last month
• hahnyuan / ASVD4LLM
  Activation-aware Singular Value Decomposition for Compressing Large Language Models
  ☆55Updated 2 months ago
• SqueezeAILab / SqueezedAttention
  SQUEEZED ATTENTION: Accelerating Long Prompt LLM Inference
  ☆36Updated last month
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆48Updated last year
• lliai / Awesome-LoRA-Low-Rank-Adaptation
  Awesome-Low-Rank-Adaptation
  ☆62Updated 3 months ago
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆27Updated 9 months ago
• zyxxmu / DSnoT
  Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLM…
  ☆38Updated 9 months ago
• ruikangliu / IntactKV
  Official PyTorch implementation of IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
  ☆38Updated 7 months ago
• pprp / Pruner-Zero
  Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆76Updated last month
• HuangOwen / RoLoRA
  [EMNLP 2024] RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization
  ☆27Updated 3 months ago
• hdong920 / LESS
  ☆49Updated 8 months ago
• ROIM1998 / APT
  [ICML'24 Oral] APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
  ☆31Updated 7 months ago
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆70Updated 7 months ago
• BaiTheBest / SparseLLM
  Official Repo for SparseLLM: Global Pruning of LLMs (NeurIPS 2024)
  ☆51Updated 3 weeks ago
• CASE-Lab-UMD / Unified-MoE-Compression
  The official implementation of the paper "Demystifying the Compression of Mixture-of-Experts Through a Unified Framework".
  ☆52Updated 2 months ago
• hdong920 / GRIFFIN
  ☆36Updated 4 months ago
• song-wx / SIFT
  [ICML2024 Spotlight] Fine-Tuning Pre-trained Large Language Models Sparsely
  ☆20Updated 6 months ago
• UNITES-Lab / moe-quantization
  Official code for the paper "Examining Post-Training Quantization for Mixture-of-Experts: A Benchmark"
  ☆10Updated 6 months ago
• CASIA-IVA-Lab / FLAP
  [AAAI 2024] Fluctuation-based Adaptive Structured Pruning for Large Language Models
  ☆40Updated last year
• VITA-Group / llm-kick
  [ICLR 2024] Jaiswal, A., Gan, Z., Du, X., Zhang, B., Wang, Z., & Yang, Y. Compressing llms: The truth is rarely pure and never simple.
  ☆19Updated 10 months ago
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆56Updated 3 months ago
• waltonfuture / Diff-eRank
  [NeurIPS 2024] A Novel Rank-Based Metric for Evaluating Large Language Models
  ☆39Updated 2 months ago