ROCm / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
☆75Updated this week
Alternatives and similar repositories for vllm:
Users that are interested in vllm are comparing it to the libraries listed below
- ☆29Updated this week
- Fast and memory-efficient exact attention☆171Updated this week
- Development repository for the Triton language and compiler☆118Updated this week
- AI Tensor Engine for ROCm☆180Updated this week
- An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).☆248Updated 5 months ago
- ☆68Updated 3 months ago
- Ahead of Time (AOT) Triton Math Library☆57Updated last week
- ☆102Updated last month
- AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…☆236Updated last week
- RCCL Performance Benchmark Tests☆64Updated this week
- ☆68Updated 3 weeks ago
- High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.☆106Updated 9 months ago
- ☆92Updated 7 months ago
- ☆198Updated 9 months ago
- ☆22Updated 2 months ago
- Ongoing research training transformer models at scale☆18Updated this week
- ☆20Updated 3 weeks ago
- Dynamic Memory Management for Serving LLMs without PagedAttention☆354Updated last week
- Fast and memory-efficient exact attention☆62Updated last week
- OpenAI Triton backend for Intel® GPUs☆182Updated this week
- ☆49Updated this week
- ☆78Updated 5 months ago
- A low-latency & high-throughput serving engine for LLMs☆346Updated last week
- High performance Transformer implementation in C++.☆118Updated 3 months ago
- Composable Kernel: Performance Portable Programming Model for Machine Learning Tensor Operators☆383Updated this week
- Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.☆61Updated 7 months ago
- Standalone Flash Attention v2 kernel without libtorch dependency☆108Updated 7 months ago
- ☆186Updated 6 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆81Updated 5 months ago
- AMD's graph optimization engine.☆215Updated this week