Alternatives and similar repositories for Awesome-LLM-Inference-Serving

Users that are interested in Awesome-LLM-Inference-Serving are comparing it to the libraries listed below

• bespokelabsai / verifiers
  Verifiers for LLM Reinforcement Learning
  ☆55Updated last month
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆36Updated last year
• Infini-AI-Lab / gsm_infinite
  ☆45Updated 3 months ago
• LLM360 / crystalcoder-data-prep
  Data preparation code for CrystalCoder 7B LLM
  ☆44Updated last year
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆55Updated last month
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆39Updated last year
• para-lost / ReBase
  ReBase: Training Task Experts through Retrieval Based Distillation
  ☆29Updated 3 months ago
• lfsszd / CS-Drafting
  Cascade Speculative Drafting
  ☆29Updated last year
• hetailang / SqueezeAttention
  ☆37Updated 7 months ago
• ContextualAI / CLAIR_and_APO
  Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment
  ☆57Updated 9 months ago
• Zyphra / Zyda_processing
  ☆34Updated 11 months ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆116Updated 5 months ago
• du-nlp-lab / MLR-Copilot
  ☆64Updated 2 months ago
• 66RING / CritiPrefill
  Code repo for "CritiPrefill: A Segment-wise Criticality-based Approach for Prefilling Acceleration in LLMs".
  ☆14Updated 8 months ago
• McGill-NLP / agent-reward-bench
  AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories
  ☆15Updated 2 weeks ago
• NathanGodey / qfilters
  Repository for the Q-Filters method (https://arxiv.org/pdf/2503.02812)
  ☆31Updated 2 months ago
• nlp-uoregon / ullme
  ☆20Updated last month
• 18907305772 / FuseAI
  FuseAI Project
  ☆87Updated 4 months ago
• The-Inscrutable-X / TACQ
  Official Repository for Task-Circuit Quantization
  ☆20Updated this week
• tanyuqian / cappy
  NeurIPS 2023 - Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
  ☆43Updated last year
• jeffreysijuntan / lloco
  The official repo for "LLoCo: Learning Long Contexts Offline"
  ☆116Updated 11 months ago
• IST-DASLab / RoSA
  Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)
  ☆42Updated last year
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆47Updated 9 months ago
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆76Updated last year
• DAMO-NLP-SG / LongPO
  [ICLR 2025] LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
  ☆36Updated 3 months ago
• ibm-granite / granite-embedding-models
  ☆28Updated 4 months ago
• eth-easl / deltazip
  Compression for Foundation Models
  ☆31Updated 2 months ago
• thunlp / APB
  Official Implementation of APB (ACL 2025 main)
  ☆28Updated 3 months ago
• arcee-ai / DAM
  ☆49Updated 6 months ago
• sail-sg / SkyLadder
  The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling
  ☆32Updated 2 months ago