A high-throughput and memory-efficient inference and serving engine for LLMs (Windows build & kernels)
☆680Oct 29, 2026Updated this week
Alternatives and similar repositories for vllm-windows
Users that are interested in vllm-windows are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- FlashInfer: Kernel Library for LLM Serving (Windows build & kernels)☆20Updated this week
- Native Windows vLLM: 0.27.1 Latest; 0.29.0 Python 3.14/CUDA 13.2 release (CPU TorchAudio), plus Python 3.13/CUDA 13.0 prerelease. PyTorch…☆64Oct 10, 2026Updated 3 weeks ago
- Patched native-Windows build of vLLM. Three Windows-specific fixes (CPU-relay for Gloo, Qwen3 reasoning parser, wildcard model name) on t…☆28May 8, 2026Updated 5 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs (Windows build & kernels)☆13Aug 15, 2026Updated 2 months ago
- Fork of the Triton language and compiler for Windows support and easy installation☆1,951Feb 18, 2026Updated 8 months ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- One-click Qwen3.6-27B inference on Windows. 158 tok/s on RTX 5090, 72 tok/s on RTX 3090. Native, no WSL, no Docker, no telemetry.☆231May 14, 2026Updated 5 months ago
- Triton with Windows support☆286Updated this week
- llama.cpp fork with additional SOTA quants and improved performance☆3,358Updated this week
- Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossi…☆160Sep 24, 2026Updated last month
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,775Updated this week
- Docker compose to run vLLM on Windows☆125Jan 1, 2024Updated 2 years ago
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,179Updated this week
- ik_llama.cpp's Thireus fork with release builds for macOS/Windows/Ubuntu CPU, Vulkan and CUDA☆183Updated this week
- Fast and memory-efficient exact attention☆940Dec 9, 2025Updated 10 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Pynini for Windows☆27Apr 11, 2025Updated last year
- ☆194Aug 30, 2026Updated 2 months ago
- Fork of SpargeAttention (SparseSageAttention) for Windows wheels and easy installation☆40Oct 9, 2026Updated 3 weeks ago
- Flash Attention WHL Builds for Windows☆16May 30, 2025Updated last year
- Fork of SageAttention for Windows wheels and easy installation☆1,065Oct 9, 2026Updated 3 weeks ago
- LLM inference in C/C++☆2,440Oct 22, 2026Updated last week
- [ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models☆3,961Sep 6, 2026Updated last month
- Official ConvRot implementation. A plug-and-play, convolution-like rotation module enabling efficient W4A4 quantization for diffusion mod…☆35Jul 3, 2026Updated 3 months ago
- Deepspeed windows information☆44Mar 9, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- KoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install.☆12,123Updated this week
- Reliable model swapping for any local AI server. Batteries included. If we don't have it, you probably don't need it.☆6,155Updated this week
- The official API server for Exllama. OAI compatible, lightweight, and fast.☆1,503Updated this week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆37,482Updated this week
- Optimized VLLM for Ampere☆24Oct 18, 2026Updated 2 weeks ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆96,053Updated this week
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,228Updated this week
- llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 30…☆57Jul 2, 2026Updated 4 months ago
- GGUF Quantization support for native ComfyUI models☆4,220Jan 12, 2026Updated 9 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Adaptive Precision for EXpert Models: MoE-aware mixed-precision quantization☆482Oct 14, 2026Updated 2 weeks ago
- Lower Precision Floating Point Operations☆86Feb 22, 2026Updated 8 months ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆185Sep 30, 2026Updated last month
- LLM inference in C/C++☆133,055Updated this week
- [ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-t…☆3,985Jan 17, 2026Updated 9 months ago
- Sage attention for turning.☆84Dec 29, 2025Updated 10 months ago
- LLM inference in C/C++☆12Oct 18, 2026Updated 2 weeks ago