Alternatives and similar repositories for ShortcutsBench:

Users that are interested in ShortcutsBench are comparing it to the libraries listed below

• UbiquitousLearning / Efficient_Foundation_Model_Survey
  Survey Paper List - Efficient LLM and Foundation Models
  ☆238Updated 4 months ago
• UbiquitousLearning / Paper-list-resource-efficient-large-language-model
  ☆99Updated last year
• zhengzangw / Sequence-Scheduling
  PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".
  ☆81Updated last year
• TreeAI-Lab / Awesome-KV-Cache-Management
  ☆57Updated last month
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆143Updated 4 months ago
• galeselee / Awesome_LLM_System-PaperList
  Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…
  ☆219Updated last month
• UChi-JCL / CacheGen
  ☆87Updated 4 months ago
• Zefan-Cai / Awesome-LLM-KV-Cache
  Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.
  ☆206Updated 2 months ago
• lambda7xx / awesome-AI-system
  paper and its code for AI System
  ☆269Updated 3 weeks ago
• October2001 / Awesome-KV-Cache-Compression
  📰 Must-read papers on KV Cache Compression (constantly updating 🤗).
  ☆292Updated last week
• hao-ai-lab / vllm-ltr
  [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank
  ☆37Updated 3 months ago
• NJUNLP / MCSD
  Multi-Candidate Speculative Decoding
  ☆34Updated 9 months ago
• tiingweii-shii / Awesome-Resource-Efficient-LLM-Papers
  a curated list of high-quality papers on resource-efficient LLMs 🌱
  ☆103Updated 2 weeks ago
• dilab-zju / self-speculative-decoding
  Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**
  ☆157Updated this week
• YaoJiayi / CacheBlend
  ☆75Updated last month
• LINs-lab / DynMoE
  [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
  ☆68Updated last week
• LiuXiaoxuanPKU / OSD
  ☆41Updated 2 months ago
• machilusZ / FastGen
  This repo contains the source code for: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
  ☆32Updated 6 months ago
• SymbioticLab / ModelKeeper
  A Cluster-Wide Model Manager to Accelerate DNN Training via Automated Training Warmup
  ☆34Updated 2 years ago
• BaiTheBest / SparseLLM
  Official Repo for SparseLLM: Global Pruning of LLMs (NeurIPS 2024)
  ☆51Updated last week
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆111Updated 11 months ago
• d-matrix-ai / keyformer-llm
  ☆52Updated 10 months ago
• mutonix / pyramidinfer
  ☆36Updated 2 months ago
• hemingkx / Spec-Bench
  Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)
  ☆222Updated 3 months ago
• yangyifei729 / KVSharer
  Source code of paper ''KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing''
  ☆23Updated 3 months ago
• xichen-fy / Fira
  Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
  ☆92Updated 3 months ago
• CASE-Lab-UMD / Unified-MoE-Compression
  The official implementation of the paper "Demystifying the Compression of Mixture-of-Experts Through a Unified Framework".
  ☆57Updated 3 months ago
• YouAreSpecialToMe / QST
  Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
  ☆41Updated 3 months ago
• interestingLSY / swiftLLM
  A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …
  ☆136Updated 7 months ago
• MobileLLM / Personal_LLM_Agents_Survey
  Paper list for Personal LLM Agents
  ☆371Updated 9 months ago