Alternatives and similar repositories for LMCache

Users that are interested in LMCache are comparing it to the libraries listed below

• ai-dynamo / dynamo
  A Datacenter Scale Distributed Inference Serving Framework
  ☆5,617Updated this week
• vllm-project / production-stack
  vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization
  ☆2,018Updated 2 weeks ago
• GeeeekExplorer / nano-vllm
  Nano vLLM
  ☆9,565Updated last month
• flashinfer-ai / flashinfer
  FlashInfer: Kernel Library for LLM Serving
  ☆4,195Updated this week
• vllm-project / aibrix
  Cost-efficient and pluggable Infrastructure components for GenAI inference
  ☆4,458Updated last week
• vllm-project / llm-compressor
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
  ☆2,348Updated last week
• kvcache-ai / Mooncake
  Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
  ☆4,399Updated this week
• predibase / lorax
  Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
  ☆3,566Updated 6 months ago
• tile-ai / tilelang
  Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels
  ☆4,113Updated last week
• THUDM / slime
  slime is an LLM post-training framework for RL Scaling.
  ☆2,816Updated this week
• ModelTC / LightLLM
  LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalabili…
  ☆3,784Updated this week
• yichuan-w / LEANN
  RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal…
  ☆4,995Updated this week
• llm-d / llm-d
  Achieve state of the art inference performance with modern accelerators on Kubernetes
  ☆2,158Updated this week
• thinking-machines-lab / tinker-cookbook
  Post-training with Tinker
  ☆2,357Updated this week
• OpenPipe / ART
  Agent Reinforcement Trainer: train multi-step agents for real-world tasks using GRPO. Give your agents on-the-job training. Reinforcement…
  ☆8,014Updated this week
• mlc-ai / xgrammar
  Fast, Flexible and Portable Structured Generation
  ☆1,418Updated this week
• Agent-on-the-Fly / Memento
  Official Code of Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
  ☆2,081Updated 2 months ago
• openai / harmony
  Renderer for the harmony response format to be used with gpt-oss
  ☆4,077Updated last month
• Mega4alik / ollm
  ☆2,213Updated 2 weeks ago
• microsoft / agent-lightning
  The absolute trainer to light up AI agents.
  ☆9,602Updated this week
• deepseek-ai / smallpond
  A lightweight data processing framework built on DuckDB and 3FS.
  ☆4,860Updated 9 months ago
• AmberLJC / LLMSys-PaperList
  Large Language Model (LLM) Systems Paper List
  ☆1,674Updated this week
• zhaochenyang20 / Awesome-ML-SYS-Tutorial
  My learning notes/codes for ML SYS.
  ☆4,374Updated this week
• deepseek-ai / EPLB
  Expert Parallelism Load Balancer
  ☆1,316Updated 8 months ago
• MoonshotAI / MoBA
  MoBA: Mixture of Block Attention for Long-Context LLMs
  ☆2,013Updated 8 months ago
• xlite-dev / Awesome-LLM-Inference
  📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉
  ☆4,791Updated 2 weeks ago
• deepseek-ai / DualPipe
  A bidirectional pipeline parallelism algorithm for computation-communication overlap in DeepSeek V3/R1 training.
  ☆2,884Updated 9 months ago
• mirage-project / mirage
  Mirage Persistent Kernel: Compiling LLMs into a MegaKernel
  ☆1,981Updated last week
• bytedance / flux
  A fast communication-overlapping library for tensor/expert parallelism on GPUs.
  ☆1,198Updated 3 months ago
• VectifyAI / PageIndex
  📑 PageIndex: Document Index for Reasoning-based RAG
  ☆4,248Updated last week