Alternatives and similar repositories for Hydra

Users that are interested in Hydra are comparing it to the libraries listed below

• FasterDecoding / TEAL
  ☆145Updated 8 months ago
• shreyansh26 / Speculative-Sampling
  Implementation of Speculative Sampling as described in "Accelerating Large Language Model Decoding with Speculative Sampling" by Deepmind
  ☆104Updated last year
• dilab-zju / self-speculative-decoding
  Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**
  ☆205Updated 8 months ago
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆169Updated last year
• FasterDecoding / SnapKV
  ☆284Updated 3 months ago
• lucidrains / speculative-decoding
  Explorations into some recent techniques surrounding speculative decoding
  ☆288Updated 10 months ago
• FasterDecoding / REST
  REST: Retrieval-Based Speculative Decoding, NAACL 2024
  ☆210Updated last month
• shawntan / scattermoe
  Triton-based implementation of Sparse Mixture of Experts.
  ☆246Updated 3 weeks ago
• RulinShao / LightSeq
  Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training
  ☆216Updated last year
• thunlp / Ouroboros
  Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)
  ☆110Updated 7 months ago
• Equationliu / Kangaroo
  [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…
  ☆60Updated last year
• kssteven418 / BigLittleDecoder
  [NeurIPS'23] Speculative Decoding with Big Little Decoder
  ☆94Updated last year
• yxli2123 / LoftQ
  ☆230Updated last year
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆64Updated last year
• UNITES-Lab / MC-SMoE
  [ICLR‘24 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆96Updated 4 months ago
• linfeng93 / BiTA
  An innovative method expediting LLMs via streamlined semi-autoregressive generation and draft verification.
  ☆26Updated 6 months ago
• AnswerDotAI / cold-compress
  Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…
  ☆147Updated last year
• hahnyuan / ASVD4LLM
  Activation-aware Singular Value Decomposition for Compressing Large Language Models
  ☆80Updated last year
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆173Updated last year
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆132Updated 2 years ago
• henryzhongsc / longctx_bench
  Official implementation for Yuan & Liu & Zhong et al., KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark o…
  ☆86Updated 7 months ago
• yaof20 / Flash-RL
  Implementation for FP8/INT8 Rollout for RL training without performence drop.
  ☆260Updated 3 weeks ago
• shadowpa0327 / Palu
  [ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projection
  ☆143Updated 8 months ago
• NVlabs / COAT
  [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training
  ☆242Updated 2 months ago
• jy-yuan / KIVI
  [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
  ☆327Updated 3 weeks ago
• HanGuo97 / lq-lora
  ☆127Updated last year
• hdong920 / GRIFFIN
  ☆38Updated last year
• Dao-AILab / grouped-latent-attention
  ☆130Updated 4 months ago
• hdong920 / LESS
  ☆52Updated last year
• jaymody / speculative-sampling
  Simple implementation of Speculative Sampling in NumPy for GPT-2.
  ☆96Updated 2 years ago