Alternatives and similar repositories for Megatrace

Users that are interested in Megatrace are comparing it to the libraries listed below

• ovg-project / kvcached
  Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
  ☆760Updated 2 weeks ago
• Multi-LLM / prism-research
  Research prototype of PRISM — a cost-efficient multi-LLM serving system with flexible time- and space-based GPU sharing.
  ☆54Updated 5 months ago
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆523Updated 4 months ago
• ai-dynamo / aiconfigurator
  Offline optimization of your disaggregated Dynamo graph
  ☆168Updated this week
• antgroup / glake
  GLake: optimizing GPU memory management and IO transmission.
  ☆497Updated 10 months ago
• sii-research / VCCL
  Venus Collective Communication Library, supported by SII and Infrawaves.
  ☆137Updated this week
• zartbot / shallowsim
  DeepSeek-V3/R1 inference performance simulator
  ☆177Updated 10 months ago
• ServerlessLLM / ServerlessLLM
  Serverless LLM Serving for Everyone.
  ☆640Updated last week
• ModelEngine-Group / unified-cache-management
  Persist and reuse KV Cache to speedup your LLM.
  ☆244Updated this week
• LLMServe / DistServe
  Disaggregated serving system for Large Language Models (LLMs).
  ☆772Updated 9 months ago
• pkusys / TGS
  Artifacts for our NSDI'23 paper TGS
  ☆94Updated last year
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆134Updated last year
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆389Updated 2 months ago
• llm-d / llm-d-inference-sim
  A light weight vLLM simulator, for mocking out replicas.
  ☆84Updated this week
• sgl-project / ome
  Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, T…
  ☆365Updated this week
• microsoft / vidur
  A large-scale simulation framework for LLM inference
  ☆527Updated 6 months ago
• S-Lab-System-Group / Awesome-DL-Scheduling-Papers
  ☆323Updated 2 years ago
• mental2008 / awesome-papers
  Here are my personal paper reading notes (including machine learning systems, AI infrastructure, and other interesting stuffs).
  ☆154Updated this week
• Bruce-Lee-LY / cuda_hook
  Hooked CUDA-related dynamic libraries by using automated code generation tools.
  ☆172Updated 2 years ago
• kvcache-ai / TrEnv-X
  ☆73Updated 4 months ago
• coreweave / nccl-tests
  NVIDIA NCCL Tests for Distributed Training
  ☆133Updated this week
• aliyun / aicb
  ☆230Updated last month
• ai-dynamo / nixl
  NVIDIA Inference Xfer Library (NIXL)
  ☆844Updated this week
• IBM / LLM-performance-prediction
  Predict the performance of LLM inference services
  ☆21Updated 4 months ago
• microsoft / NPKit
  NCCL Profiling Kit
  ☆150Updated last year
• SJTU-IPADS / PhoenixOS
  Fast OS-level support for GPU checkpoint and restore
  ☆270Updated 4 months ago
• WukLab / preble
  Stateful LLM Serving
  ☆95Updated 10 months ago
• UChi-JCL / CacheGen
  ☆147Updated last year
• sgl-project / rbg
  A workload for deploying LLM inference services on Kubernetes
  ☆160Updated last week
• InternLM / AcmeTrace
  ☆174Updated last year