Alternatives and similar repositories for decoding-speculative-decoding:

Users that are interested in decoding-speculative-decoding are comparing it to the libraries listed below

• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆38Updated 11 months ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆107Updated 2 months ago
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆40Updated last year
• INT-FlashAttention2024 / INT-FlashAttention
  ☆61Updated 3 weeks ago
• IST-DASLab / QIGen
  Repository for CPU Kernel Generation for LLM Inference
  ☆25Updated last year
• ScalingIntelligence / CATS
  ☆23Updated 3 months ago
• Raincleared-Song / sparse_gpu_operator
  GPU operators for sparse tensor operations
  ☆30Updated 11 months ago
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆64Updated 5 months ago
• linfeng93 / BiTA
  An innovative method expediting LLMs via streamlined semi-autoregressive generation and draft verification.
  ☆24Updated last year
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆56Updated 4 months ago
• MDK8888 / vllmini
  A minimal implementation of vllm.
  ☆33Updated 6 months ago
• pprp / Pruner-Zero
  Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆78Updated 2 months ago
• hdong920 / GRIFFIN
  ☆36Updated 5 months ago
• sail-sg / SimLayerKV
  The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.
  ☆42Updated 4 months ago
• DerrickYLJ / TidalDecode
  [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
  ☆27Updated this week
• Equationliu / Kangaroo
  [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…
  ☆48Updated 7 months ago
• hetailang / SqueezeAttention
  ☆37Updated 4 months ago
• PipeFusion / PipeFusion
  A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters
  ☆39Updated 6 months ago
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆28Updated 8 months ago
• d-matrix-ai / keyformer-llm
  ☆52Updated 10 months ago
• feifeibear / ChituAttention
  Quantized Attention on GPU
  ☆34Updated 3 months ago
• Qualcomm-AI-research / gptvq
  ☆27Updated 10 months ago
• sail-sg / VocabularyParallelism
  Vocabulary Parallelism
  ☆17Updated 3 months ago
• BaiTheBest / SparseLLM
  Official Repo for SparseLLM: Global Pruning of LLMs (NeurIPS 2024)
  ☆51Updated last week
• NVlabs / COAT
  ☆87Updated this week
• mit-han-lab / Quest
  [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
  ☆245Updated 2 months ago
• smart-lty / ParallelSpeculativeDecoding
  [ICLR 2025] PEARL: parallel speculative decoding with adaptive draft length
  ☆39Updated this week
• hahnyuan / ASVD4LLM
  Activation-aware Singular Value Decomposition for Compressing Large Language Models
  ☆56Updated 4 months ago
• zankner / Hydra
  ☆40Updated last year