Alternatives and similar repositories for triton_distributed:

Users that are interested in triton_distributed are comparing it to the libraries listed below

• ai-dynamo / nixl
  NVIDIA Inference Xfer Library (NIXL)
  ☆282Updated this week
• NVIDIA / nvidia-resiliency-ext
  NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …
  ☆150Updated this week
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆383Updated this week
• coreweave / nccl-tests
  NVIDIA NCCL Tests for Distributed Training
  ☆88Updated this week
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆346Updated this week
• google / nccl-fastsocket
  NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.
  ☆116Updated last year
• triton-inference-server / perf_analyzer
  ☆61Updated this week
• microsoft / mscclpp
  MSCCL++: A GPU-driven communication stack for scalable AI applications
  ☆341Updated this week
• yifuwang / symm-mem-recipes
  ☆66Updated 3 months ago
• microsoft / NPKit
  NCCL Profiling Kit
  ☆130Updated 9 months ago
• Azure / msccl-executor-nccl
  ☆36Updated 4 months ago
• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆348Updated this week
• Mellanox / nccl-rdma-sharp-plugins
  RDMA and SHARP plugins for nccl library
  ☆189Updated 2 weeks ago
• microsoft / msccl
  Microsoft Collective Communication Library
  ☆343Updated last year
• ppl-ai / pplx-kernels
  Perplexity GPU Kernels
  ☆235Updated 2 weeks ago
• bytedance-iaas / splitwise-demos
  ☆23Updated last month
• LLMServe / SwiftTransformer
  High performance Transformer implementation in C++.
  ☆118Updated 3 months ago
• ColfaxResearch / cutlass-kernels
  ☆198Updated 9 months ago
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆151Updated 3 weeks ago
• eniac / paella
  Paella: Low-latency Model Serving with Virtualized GPU Scheduling
  ☆58Updated 11 months ago
• AlibabaPAI / torchacc
  PyTorch distributed training acceleration framework
  ☆48Updated 2 months ago
• LoongServe / LoongServe
  ☆95Updated 5 months ago
• ColfaxResearch / cfx-article-src
  ☆102Updated last month
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆77Updated 2 months ago
• sail-sg / zero-bubble-pipeline-parallelism
  Zero Bubble Pipeline Parallelism
  ☆383Updated 2 weeks ago
• triton-inference-server / backend
  Common source, scripts and utilities for creating Triton backends.
  ☆316Updated last week
• microsoft / nnscaler
  nnScaler: Compiling DNN models for Parallel Training
  ☆107Updated this week
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆261Updated last month
• neuralmagic / AutoFP8
  ☆186Updated 6 months ago
• OpenPPL / ppl.nn.llm
  ☆139Updated last year