ai-dynamo / nixl
NVIDIA Inference Xfer Library (NIXL)
☆304Updated this week
Alternatives and similar repositories for nixl:
Users that are interested in nixl are comparing it to the libraries listed below
- Efficient and easy multi-instance LLM serving☆398Updated this week
- Perplexity GPU Kernels☆272Updated this week
- Dynamic Memory Management for Serving LLMs without PagedAttention☆360Updated 2 weeks ago
- ☆49Updated last month
- MSCCL++: A GPU-driven communication stack for scalable AI applications☆345Updated this week
- A low-latency & high-throughput serving engine for LLMs☆351Updated 2 weeks ago
- NCCL Profiling Kit☆133Updated 10 months ago
- KV cache store for distributed LLM inference☆165Updated this week
- Disaggregated serving system for Large Language Models (LLMs).☆575Updated 3 weeks ago
- Zero Bubble Pipeline Parallelism☆386Updated 3 weeks ago
- Microsoft Collective Communication Library☆344Updated last year
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆151Updated this week
- NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.☆116Updated last year
- DeepSeek-V3/R1 inference performance simulator☆115Updated last month
- Distributed Triton for Parallel Systems☆618Updated last week
- CUDA checkpoint and restore utility☆330Updated 3 months ago
- High performance Transformer implementation in C++.☆119Updated 3 months ago
- A library to analyze PyTorch traces.☆367Updated last week
- A tool for bandwidth measurements on NVIDIA GPUs.☆413Updated 3 weeks ago
- RDMA and SHARP plugins for nccl library☆191Updated 3 weeks ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆64Updated this week
- ☆202Updated 9 months ago
- Materials for learning SGLang☆396Updated last week
- NVIDIA NCCL Tests for Distributed Training☆88Updated last week
- ☆36Updated 4 months ago
- GLake: optimizing GPU memory management and IO transmission.☆456Updated last month
- Synthesizer for optimal collective communication algorithms☆106Updated last year
- AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…☆238Updated 2 weeks ago
- Paella: Low-latency Model Serving with Virtualized GPU Scheduling☆58Updated last year
- A throughput-oriented high-performance serving framework for LLMs☆804Updated this week