triton-inference-server / triton_distributed
☆49Updated 2 months ago
Alternatives and similar repositories for triton_distributed
Users that are interested in triton_distributed are comparing it to the libraries listed below
Sorting:
- NVIDIA Inference Xfer Library (NIXL)☆324Updated this week
- Efficient and easy multi-instance LLM serving☆404Updated this week
- Perplexity GPU Kernels☆281Updated 2 weeks ago
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆159Updated this week
- Dynamic Memory Management for Serving LLMs without PagedAttention☆366Updated 3 weeks ago
- MSCCL++: A GPU-driven communication stack for scalable AI applications☆349Updated this week
- NVIDIA NCCL Tests for Distributed Training☆90Updated last week
- ☆25Updated 2 months ago
- KV cache store for distributed LLM inference☆190Updated this week
- NCCL Profiling Kit☆133Updated 10 months ago
- A low-latency & high-throughput serving engine for LLMs☆360Updated 3 weeks ago
- NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.☆116Updated last year
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆73Updated 2 weeks ago
- ☆69Updated this week
- AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…☆239Updated 3 weeks ago
- Experimental projects related to TensorRT☆99Updated this week
- ☆72Updated 4 months ago
- ☆202Updated 10 months ago
- Microsoft Collective Communication Library☆345Updated last year
- Microsoft Collective Communication Library☆65Updated 5 months ago
- GLake: optimizing GPU memory management and IO transmission.☆460Updated last month
- A lightweight design for computation-communication overlap.☆92Updated last week
- Zero Bubble Pipeline Parallelism☆389Updated last week
- RDMA and SHARP plugins for nccl library☆193Updated last month
- DeepSeek-V3/R1 inference performance simulator☆120Updated last month
- PyTorch distributed training acceleration framework☆48Updated 3 months ago
- ☆109Updated last week
- Paella: Low-latency Model Serving with Virtualized GPU Scheduling☆58Updated last year
- High performance Transformer implementation in C++.☆122Updated 3 months ago
- ☆36Updated 5 months ago