vllm-project / vllm-nccl
Manages vllm-nccl dependency
☆17Updated 9 months ago
Alternatives and similar repositories for vllm-nccl:
Users that are interested in vllm-nccl are comparing it to the libraries listed below
- Vocabulary Parallelism☆17Updated this week
- Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang☆40Updated 4 months ago
- Distributed IO-aware Attention algorithm☆18Updated 6 months ago
- [ICLR 2025] PEARL: parallel speculative decoding with adaptive draft length☆52Updated this week
- Odysseus: Playground of LLM Sequence Parallelism☆66Updated 8 months ago
- Summary of system papers/frameworks/codes/tools on training or serving large model☆56Updated last year
- Repository of LV-Eval Benchmark☆59Updated 6 months ago
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆39Updated last year
- Transformer related optimization, including BERT, GPT☆17Updated last year
- A MoE impl for PyTorch, [ATC'23] SmartMoE☆61Updated last year
- Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)☆28Updated last year
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)☆85Updated 5 months ago
- [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejection☆39Updated 4 months ago
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆36Updated last year
- Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.☆34Updated this week
- The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.☆43Updated 4 months ago
- ☆19Updated last month
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆93Updated last year
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆128Updated 9 months ago
- ☆78Updated last year
- ☆14Updated last year
- A more efficient GLM implementation!☆55Updated 2 years ago
- ☆22Updated last year
- ☆63Updated 3 months ago
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆40Updated last year
- Sequence-level 1F1B schedule for LLMs.☆17Updated 9 months ago
- Elixir: Train a Large Language Model on a Small GPU Cluster☆13Updated last year
- GPTQ inference TVM kernel☆39Updated 10 months ago
- Transformer related optimization, including BERT, GPT☆39Updated 2 years ago