Alternatives and similar repositories for triton_distributed:

Users that are interested in triton_distributed are comparing it to the libraries listed below

• ai-dynamo / nixl
  NVIDIA Inference Xfer Library (NIXL)
  ☆191Updated this week
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆348Updated this week
• coreweave / nccl-tests
  NVIDIA NCCL Tests for Distributed Training
  ☆85Updated last week
• microsoft / mscclpp
  MSCCL++: A GPU-driven communication stack for scalable AI applications
  ☆315Updated this week
• NVIDIA / nvidia-resiliency-ext
  NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …
  ☆109Updated this week
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆327Updated last month
• NVIDIA / cuda-checkpoint
  CUDA checkpoint and restore utility
  ☆315Updated 2 months ago
• NVIDIA / nvbandwidth
  A tool for bandwidth measurements on NVIDIA GPUs.
  ☆392Updated last month
• triton-inference-server / perf_analyzer
  ☆51Updated this week
• Mellanox / nccl-rdma-sharp-plugins
  RDMA and SHARP plugins for nccl library
  ☆184Updated this week
• google / nccl-fastsocket
  NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.
  ☆116Updated last year
• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆326Updated this week
• Bruce-Lee-LY / cuda_hook
  Hooked CUDA-related dynamic libraries by using automated code generation tools.
  ☆150Updated last year
• yifuwang / symm-mem-recipes
  ☆57Updated 3 months ago
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆62Updated this week
• microsoft / NPKit
  NCCL Profiling Kit
  ☆128Updated 8 months ago
• NVIDIA / TensorRT-Incubator
  Experimental projects related to TensorRT
  ☆94Updated this week
• facebookresearch / HolisticTraceAnalysis
  A library to analyze PyTorch traces.
  ☆350Updated last week
• triton-inference-server / model_analyzer
  Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Serv…
  ☆462Updated 2 weeks ago
• Azure / msccl
  Microsoft Collective Communication Library
  ☆60Updated 4 months ago
• microsoft / msccl
  Microsoft Collective Communication Library
  ☆343Updated last year
• bytedance / ByteMLPerf
  AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…
  ☆233Updated this week
• triton-inference-server / core
  The core library and APIs implementing the Triton Inference Server.
  ☆123Updated this week
• NVIDIA-Merlin / HierarchicalKV
  HierarchicalKV is a part of NVIDIA Merlin and provides hierarchical key-value storage to meet RecSys requirements. The key capability of…
  ☆141Updated 3 weeks ago
• imbue-ai / cluster-health
  ☆296Updated 7 months ago
• LLMServe / DistServe
  Disaggregated serving system for Large Language Models (LLMs).
  ☆507Updated 7 months ago
• triton-inference-server / backend
  Common source, scripts and utilities for creating Triton backends.
  ☆311Updated this week
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆250Updated last week
• alibaba / TePDist
  TePDist (TEnsor Program DISTributed) is an HLO-level automatic distributed system for DL models.
  ☆92Updated last year
• antgroup / glake
  GLake: optimizing GPU memory management and IO transmission.
  ☆449Updated this week