vllm-project / vllm-ncclLinks
Manages vllm-nccl dependency
☆17Updated last year
Alternatives and similar repositories for vllm-nccl
Users that are interested in vllm-nccl are comparing it to the libraries listed below
Sorting:
- Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang☆56Updated 9 months ago
- Vocabulary Parallelism☆20Updated 5 months ago
- Open deep learning compiler stack for cpu, gpu and specialized accelerators☆19Updated last week
- Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)☆28Updated 2 years ago
- TensorRT LLM Benchmark Configuration☆13Updated last year
- Summary of system papers/frameworks/codes/tools on training or serving large model☆57Updated last year
- ☆79Updated last year
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- Transformers components but in Triton☆34Updated 3 months ago
- ☆121Updated last year
- 方便扩展的Cuda算子理解和优化框架,仅用在学习使用☆16Updated last year
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆41Updated last year
- GPTQ inference TVM kernel☆40Updated last year
- Inference framework for MoE layers based on TensorRT with Python binding☆41Updated 4 years ago
- LLMem: GPU Memory Estimation for Fine-Tuning Pre-Trained LLMs☆22Updated 3 months ago
- Elixir: Train a Large Language Model on a Small GPU Cluster☆15Updated 2 years ago
- ☆20Updated 4 months ago
- Official Implementation of APB (ACL 2025 main Oral)☆31Updated 6 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆125Updated 8 months ago
- Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.☆40Updated 2 months ago
- ☆18Updated this week
- ☆41Updated 4 months ago
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)☆109Updated 5 months ago
- Repository for CPU Kernel Generation for LLM Inference☆26Updated 2 years ago
- Odysseus: Playground of LLM Sequence Parallelism☆76Updated last year
- ☆74Updated 5 months ago
- The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.☆48Updated 10 months ago
- An easy-to-use package for implementing SmoothQuant for LLMs☆105Updated 4 months ago
- ☆92Updated 5 months ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆136Updated last year