Alternatives and similar repositories for ninetoothed

Users that are interested in ninetoothed are comparing it to the libraries listed below

• zartbot / shallowsim
  DeepSeek-V3/R1 inference performance simulator
  ☆154Updated 3 months ago
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆168Updated 9 months ago
• CalvinXKY / mfu_calculation
  A simple calculation for LLM MFU.
  ☆39Updated 4 months ago
• infinigence / Semi-PD
  A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
  ☆95Updated last month
• hao-ai-lab / MuxServe
  ☆64Updated last year
• infinigence / FlashOverlap
  A lightweight design for computation-communication overlap.
  ☆146Updated 3 weeks ago
• InfiniTensor / InfiniLM
  ☆117Updated this week
• WukLab / preble
  Stateful LLM Serving
  ☆76Updated 4 months ago
• cassiewilliam / cuda_op_benchmark
  方便扩展的Cuda算子理解和优化框架，仅用在学习使用
  ☆15Updated last year
• PKU-SEC-Lab / HybriMoE
  [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"
  ☆58Updated last month
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆69Updated 2 months ago
• thunlp / TritonBench
  TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators
  ☆62Updated last month
• SJTU-IPADS / PhoenixOS
  Fast OS-level support for GPU checkpoint and restore
  ☆212Updated 3 weeks ago
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆79Updated 4 months ago
• pku-liang / ArkVale
  ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction (NIPS'24)
  ☆41Updated 6 months ago
• InfiniTensor / InfiniTensor
  ☆237Updated last month
• LoongServe / LoongServe
  ☆106Updated 8 months ago
• shenh10 / DeepSeek_Simulator
  ☆79Updated 3 months ago
• Infrawaves / DeepEP_ibrc_dual-ports_multiQP
  Aims to implement dual-port and multi-qp solutions in deepEP ibrc transport
  ☆53Updated 2 months ago
• flashinfer-ai / cutlass-viz
  ☆60Updated 2 months ago
• microsoft / TileFusion
  TileFusion is an experimental C++ macro kernel template library that elevates the abstraction level in CUDA C for tile processing.
  ☆90Updated 2 weeks ago
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆44Updated 5 months ago
• tsinghua-ideal / Canvas
  Canvas: End-to-End Kernel Architecture Search in Neural Networks
  ☆27Updated 7 months ago
• sjtu-epcc / Tacker
  Tacker: Tensor-CUDA Core Kernel Fusion for Improving the GPU Utilization while Ensuring QoS
  ☆27Updated 5 months ago
• NEO-MLSys25 / NEO
  NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading
  ☆44Updated 3 weeks ago
• SymbioticLab / Oobleck
  A resilient distributed training framework
  ☆95Updated last year
• DerrickYLJ / TidalDecode
  [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
  ☆40Updated 2 months ago
• HPMLL / NVIDIA-Hopper-Benchmark
  ☆49Updated last month
• microsoft / FractalTensor
  FractalTensor is a programming framework that introduces a novel approach to organizing data in deep neural networks (DNNs) as a list of …
  ☆27Updated 6 months ago
• abhibambhaniya / GenZ-LLM-Analyzer
  LLM Inference analyzer for different hardware platforms
  ☆79Updated last week