Alternatives and similar repositories for LatencySensitiveBench

Users that are interested in LatencySensitiveBench are comparing it to the libraries listed below

• Jiacheng-Zhu-AIML / AsymmetryLoRA
  Preprint: Asymmetry in Low-Rank Adapters of Foundation Models
  ☆35Updated last year
• SqueezeAILab / SqueezedAttention
  SQUEEZED ATTENTION: Accelerating Long Prompt LLM Inference
  ☆47Updated 7 months ago
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆16Updated 2 months ago
• alvin-zyl / CoLA
  Implementation of CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
  ☆22Updated 4 months ago
• guyuntian / CoT_benchmark
  Code for "Towards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective"
  ☆20Updated last year
• Infini-AI-Lab / Sirius
  Sirius, an efficient correction mechanism, which significantly boosts Contextual Sparsity models on reasoning tasks while maintaining its…
  ☆22Updated 9 months ago
• sail-sg / Attention-Sink
  [ICLR 2025] When Attention Sink Emerges in Language Models: An Empirical View (Spotlight)
  ☆88Updated 8 months ago
• AkideLiu / MiniCache
  ☆9Updated 9 months ago
• sail-sg / lm-random-memory-access
  ☆14Updated last year
• LiuXiaoxuanPKU / OSD
  ☆52Updated 6 months ago
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆57Updated last week
• yxli2123 / LoSparse
  ☆57Updated last year
• ryoungj / BoLT
  Code for "Reasoning to Learn from Latent Thoughts"
  ☆105Updated 2 months ago
• Jingyu6 / speculative_prefill
  ☆30Updated last month
• sail-sg / LongSpec
  LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
  ☆54Updated 3 months ago
• hdong920 / LESS
  ☆50Updated last year
• zyxxmu / cam
  Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference
  ☆39Updated last year
• imagination-research / EEP
  Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
  ☆19Updated 6 months ago
• bethgelab / sober-reasoning
  A Sober Look at Language Model Reasoning
  ☆74Updated last week
• mingluzhao / Latent-Plan-Transformer
  Source code for "Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inference." In NeurIPS 2024
  ☆16Updated 6 months ago
• Lucky-Lance / SPP
  [ICML 2024] SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
  ☆21Updated last year
• machilusZ / FastGen
  This repo contains the source code for: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
  ☆38Updated 10 months ago
• qiuzh20 / gated_attention
  The official implementation for Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
  ☆44Updated last month
• abhishekpanigrahi1996 / Skill-Localization-by-grafting
  ☆49Updated last year
• BunsenFeng / model_swarm
  ☆17Updated 6 months ago
• thu-wyz / inference_scaling
  ☆71Updated 7 months ago
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆60Updated 9 months ago
• QingruZhang / PLATON
  This pytorch package implements PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance (ICML 2022).
  ☆46Updated 2 years ago
• PKU-ML / LongPPL
  Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"
  ☆88Updated last month
• tilde-research / nsa-impl
  An efficient implementation of the NSA (Native Sparse Attention) kernel
  ☆72Updated this week