ROCm / vllmLinks
A high-throughput and memory-efficient inference and serving engine for LLMs
☆108Updated this week
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below
Sorting:
- Development repository for the Triton language and compiler☆135Updated this week
- Fast and memory-efficient exact attention☆193Updated this week
- ☆48Updated this week
- AI Tensor Engine for ROCm☆285Updated last week
- Ahead of Time (AOT) Triton Math Library☆79Updated this week
- OpenAI Triton backend for Intel® GPUs☆211Updated this week
- Fast and memory-efficient exact attention☆96Updated last week
- An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).☆265Updated 3 months ago
- RCCL Performance Benchmark Tests☆75Updated last week
- QuickReduce is a performant all-reduce library designed for AMD ROCm that supports inline compression.☆33Updated last month
- ☆124Updated 9 months ago
- ☆91Updated this week
- ☆100Updated last year
- ☆205Updated 5 months ago
- High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.☆116Updated last year
- Ongoing research training transformer models at scale☆29Updated this week
- AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…☆265Updated 2 months ago
- Composable Kernel: Performance Portable Programming Model for Machine Learning Tensor Operators☆472Updated this week
- QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.☆144Updated 2 months ago
- A lightweight design for computation-communication overlap.☆181Updated last week
- ☆129Updated 9 months ago
- ☆33Updated 8 months ago
- FlexFlow Serve: Low-Latency, High-Performance LLM Serving☆63Updated last month
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆115Updated last year
- PyTorch distributed training acceleration framework☆53Updated 2 months ago
- ☆27Updated 3 weeks ago
- A safetensors extension to efficiently store sparse quantized tensors on disk☆171Updated this week
- ☆92Updated 11 months ago
- A tool for bandwidth measurements on NVIDIA GPUs.☆550Updated 6 months ago
- ☆46Updated 10 months ago