Alternatives and similar repositories for Mooncake:

Users that are interested in Mooncake are comparing it to the libraries listed below

• flashinfer-ai / flashinfer
  FlashInfer: Kernel Library for LLM Serving
  ☆2,111Updated this week
• ModelTC / lightllm
  LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalabili…
  ☆2,911Updated this week
• DefTruth / Awesome-LLM-Inference
  📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, Flash-Attention, Paged-Attention, Parallelism, etc. 🎉🎉
  ☆3,456Updated this week
• alibaba / rtp-llm
  RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
  ☆629Updated last month
• InternLM / lmdeploy
  LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
  ☆5,630Updated this week
• zhaochenyang20 / Awesome-ML-SYS-Tutorial
  My learning notes/codes for ML SYS.
  ☆822Updated this week
• sgl-project / sglang
  SGLang is a fast serving framework for large language models and vision language models.
  ☆10,325Updated this week
• OpenRLHF / OpenRLHF
  An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention & RFT)
  ☆4,895Updated this week
• alibaba / Pai-Megatron-Patch
  The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.
  ☆856Updated last week
• volcengine / veScale
  A PyTorch Native LLM Training Framework
  ☆732Updated last month
• intelligent-machine-learning / dlrover
  DLRover: An Automatic Distributed Deep Learning System
  ☆1,342Updated this week
• open-compass / opencompass
  OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, …
  ☆4,705Updated this week
• deepseek-ai / DeepSeek-MoE
  DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
  ☆1,409Updated last year
• DefTruth / CUDA-Learn-Notes
  📚200+ Tensor/CUDA Cores Kernels, ⚡️flash-attn-mma, ⚡️hgemm with WMMA, MMA and CuTe (98%~100% TFLOPS of cuBLAS/FA2 🎉🎉).
  ☆2,365Updated this week
• pcg-mlp / KsanaLLM
  ☆314Updated last month
• openreasoner / openr
  OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
  ☆1,651Updated last month
• volcengine / verl
  verl: Volcano Engine Reinforcement Learning for LLMs
  ☆3,513Updated this week
• harleyszhang / llm_note
  LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.
  ☆494Updated this week
• GAIR-NLP / O1-Journey
  O1 Replication Journey
  ☆1,947Updated last month
• InternLM / xtuner
  An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)
  ☆4,246Updated 3 weeks ago
• modelscope / evalscope
  A streamlined and customizable framework for efficient large model evaluation and performance benchmarking
  ☆442Updated this week
• deepspeedai / Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆1,990Updated 2 weeks ago
• mit-han-lab / llm-awq
  [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
  ☆2,760Updated last week
• microsoft / MInference
  [NeurIPS'24 Spotlight, ICLR'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which r…
  ☆917Updated last week
• alibaba / Megatron-LLaMA
  Best practice for training LLaMA models in Megatron-LM
  ☆644Updated last year
• QwenLM / Qwen-Agent
  Agent framework and applications built upon Qwen>=2.0, featuring Function Calling, Code Interpreter, RAG, and Chrome extension.
  ☆5,852Updated 3 weeks ago
• charent / ChatLM-mini-Chinese
  中文对话0.2B小模型（ChatLM-Chinese-0.2B），开源所有数据集来源、数据清洗、tokenizer训练、模型预训练、SFT指令微调、RLHF优化等流程的全部代码。支持下游任务sft微调，给出三元组信息抽取微调示例。
  ☆1,410Updated 10 months ago
• antgroup / glake
  GLake: optimizing GPU memory management and IO transmission.
  ☆431Updated 2 months ago