Alternatives and similar repositories for ShortcutsBench:

Users that are interested in ShortcutsBench are comparing it to the libraries listed below

• UbiquitousLearning / Paper-list-resource-efficient-large-language-model
  ☆99Updated last year
• zhengzangw / Sequence-Scheduling
  PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".
  ☆81Updated last year
• UbiquitousLearning / Efficient_Foundation_Model_Survey
  Survey Paper List - Efficient LLM and Foundation Models
  ☆238Updated 3 months ago
• UChi-JCL / CacheGen
  ☆81Updated 3 months ago
• October2001 / Awesome-KV-Cache-Compression
  📰 Must-read papers on KV Cache Compression (constantly updating 🤗).
  ☆260Updated this week
• tiingweii-shii / Awesome-Resource-Efficient-LLM-Papers
  a curated list of high-quality papers on resource-efficient LLMs 🌱
  ☆93Updated 2 weeks ago
• Zefan-Cai / Awesome-LLM-KV-Cache
  Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.
  ☆191Updated last month
• NJUNLP / MCSD
  Multi-Candidate Speculative Decoding
  ☆34Updated 8 months ago
• MobileLLM / ChainStream
  A Stream-based LLM Agent Framework for Continuous Context Sensing and Sharing
  ☆35Updated last month
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆134Updated 3 months ago
• TreeAI-Lab / Awesome-KV-Cache-Management
  ☆43Updated 2 weeks ago
• SkyworkAI / skywork-o1-prm-inference
  ☆50Updated last month
• xichen-fy / Fira
  Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
  ☆89Updated 3 months ago
• hemingkx / Spec-Bench
  Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)
  ☆214Updated 2 months ago
• hdong920 / GRIFFIN
  ☆36Updated 4 months ago
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆70Updated 7 months ago
• interestingLSY / swiftLLM
  A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …
  ☆132Updated 6 months ago
• LlamaTouch / LlamaTouch
  LlamaTouch: A Faithful and Scalable Testbed for Mobile UI Task Automation
  ☆53Updated 5 months ago
• henryzhongsc / longctx_bench
  Official implementation for Yuan & Liu & Zhong et al., KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark o…
  ☆61Updated 3 weeks ago
• dilab-zju / self-speculative-decoding
  Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**
  ☆151Updated 7 months ago
• hyx1999 / SAM-Decoding
  Official Implementation of SAM-Decoding: Speculative Decoding via Suffix Automaton
  ☆16Updated 3 weeks ago
• Relaxed-System-Lab / HexGen
  [ICML 2024] Serving LLMs on heterogeneous decentralized clusters.
  ☆17Updated 8 months ago
• hao-ai-lab / vllm-ltr
  [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank
  ☆29Updated 2 months ago
• LINs-lab / DynMoE
  [Preprint] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
  ☆59Updated 4 months ago
• hemingkx / SWIFT
  SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
  ☆31Updated last month
• getao / icae
  The repo for In-context Autoencoder
  ☆103Updated 8 months ago
• LiuXiaoxuanPKU / OSD
  ☆40Updated last month
• galeselee / Awesome_LLM_System-PaperList
  Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…
  ☆205Updated 3 weeks ago
• HarryWu99 / llm_kvcache_sparsity
  Implement some method of LLM KV Cache Sparsity
  ☆30Updated 7 months ago
• Relaxed-System-Lab / COMP4901Y_Course_HKUST
  Course Material for the UG Course COMP4901Y
  ☆50Updated 8 months ago