Alternatives and similar repositories for xllm-service

Users that are interested in xllm-service are comparing it to the libraries listed below

• alibaba / Pai-Megatron-Patch
  The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.
  ☆1,316Updated last week
• alibaba / ROLL
  An Efficient and User-Friendly Scaling Library for Reinforcement Learning with Large Language Models
  ☆1,862Updated this week
• PKU-DAIR / Hetu
  A high-performance distributed deep learning system targeting large-scale and automated distributed training.
  ☆320Updated last month
• alibaba / Megatron-LLaMA
  Best practice for training LLaMA models in Megatron-LM
  ☆660Updated last year
• jd-opensource / xllm
  A high-performance inference engine for LLMs, optimized for diverse AI accelerators.
  ☆199Updated this week
• LLMServe / DistServe
  Disaggregated serving system for Large Language Models (LLMs).
  ☆683Updated 4 months ago
• Ketonmi / Awesome-Large-Scale-LLM-Serving
  Must-read papers on improving efficiency for LLM serving clusters
  ☆31Updated 3 months ago
• alibaba / rtp-llm
  RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
  ☆844Updated last month
• intelligent-machine-learning / dlrover
  DLRover: An Automatic Distributed Deep Learning System
  ☆1,533Updated last week
• FlagOpen / FlagScale
  FlagScale is a large model toolkit based on open-sourced projects.
  ☆349Updated this week
• vllm-project / vllm-ascend
  Community maintained hardware plugin for vLLM on Ascend
  ☆1,067Updated this week
• alibaba / ChatLearn
  A flexible and efficient training framework for large-scale alignment tasks
  ☆416Updated this week
• feifeibear / LLMSpeculativeSampling
  Fast inference from large lauguage models via speculative decoding
  ☆811Updated last year
• antgroup / cakekv
  ☆25Updated 5 months ago
• hemingkx / SpeculativeDecodingPapers
  📰 Must-read papers and blogs on Speculative Decoding ⚡️
  ☆906Updated this week
• AFDWang / Hetu-Galvatron
  Galvatron is an automatic distributed training system designed for Transformer models, including Large Language Models (LLMs). If you hav…
  ☆22Updated 5 months ago
• TUDB-Labs / mLoRA
  An Efficient "Factory" to Build Multiple LoRA Adapters
  ☆337Updated 6 months ago
• TreeAI-Lab / Awesome-KV-Cache-Management
  This repository serves as a comprehensive survey of LLM development, featuring numerous research papers along with their corresponding co…
  ☆192Updated last month
• THUDM / LongBench
  LongBench v2 and LongBench (ACL 25'&24')
  ☆953Updated 7 months ago
• Zefan-Cai / Awesome-LLM-KV-Cache
  Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.
  ☆356Updated 6 months ago
• NVIDIA / recsys-examples
  Examples for Recommenders - easy to train and deploy on accelerated infrastructure.
  ☆124Updated this week
• zhaochenyang20 / Awesome-ML-SYS-Tutorial
  My learning notes/codes for ML SYS.
  ☆3,515Updated this week
• volcengine / veScale
  A PyTorch Native LLM Training Framework
  ☆861Updated last month
• October2001 / Awesome-KV-Cache-Compression
  📰 Must-read papers on KV Cache Compression (constantly updating 🤗).
  ☆525Updated last month
• FMInference / H2O
  [NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.
  ☆471Updated last year
• bytedance / HLLM
  HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling
  ☆475Updated last week
• AmadeusChan / Awesome-LLM-System-Papers
  ☆608Updated 3 months ago
• hemingkx / Spec-Bench
  Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)
  ☆309Updated 4 months ago
• NoakLiu / DRTR
  Adaptive Topology Reconstruction for Robust Graph Representation Learning [Efficient ML Model]
  ☆10Updated 6 months ago
• microsoft / vidur
  A large-scale simulation framework for LLM inference
  ☆428Updated last month