A high-throughput and memory-efficient inference and serving engine for LLMs
☆41Aug 13, 2026Updated 2 months ago
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Qwen3.8-Flash-Next 180B NVFP4 @ 231 tok/s on one RTX PRO 6000 Blackwell (sm120) - sglang patches, launchers, full optimization analysis☆30Aug 31, 2026Updated 2 months ago
- ☆75Feb 27, 2026Updated 8 months ago
- Implementation of BLAS routines in pure Mojo 🔥☆13Sep 4, 2026Updated last month
- The complete NUMA-optimized branch of the ktransformers project☆27Nov 3, 2025Updated last year
- Mixed-precision quantization for LLMs. Every layer refracts into a different format based on its sensitivity. Native compressed-tensors e…☆105Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Notification integration for SimpleX messenger☆23Oct 19, 2026Updated 2 weeks ago
- ☆120Aug 6, 2026Updated 2 months ago
- LLM inference decode throughput benchmark with Rich TUI dashboard. Measures token generation speed across concurrency levels and context …☆121Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆13Apr 14, 2026Updated 6 months ago
- ☆17Aug 13, 2026Updated 2 months ago
- GLM-5.3-Flash NVFP4 (lab quant) on 2x DGX Spark — TP=2 over RoCE, one .env, compose head/worker, download/start/stop/status/tail-log☆11Sep 8, 2026Updated last month
- Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.☆20Aug 10, 2026Updated 2 months ago
- ☆26Updated this week
- GLM-5.3 Flash EXL3 for 2-4x DGX Sparks☆747Oct 18, 2026Updated 2 weeks ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- GLM-5.3-Flash NVFP4 on 4x NVIDIA DGX Spark: vLLM TP4, DFlash2 adaptive draft, measured FP8 vs NVFP4 vs EXL3 vs SGLang☆40Sep 29, 2026Updated last month
- 让AI使用的Dify-WorkFlow-CLI工具☆21Apr 8, 2026Updated 6 months ago
- DSH for Teams — Deploy DSH once on your LAN; memory, knowledge base & document search included. Team-ready, up to ~20 users. MIT-licensed…☆10Updated this week
- antirez/ds4-style hybrid quant DeepSeek V4 Flash on a single DGX Spark via vLLM☆17May 11, 2026Updated 5 months ago
- Archived: b12x development has moved to FlashInfer. See README for migration and PR-porting guidance.☆268Oct 6, 2026Updated 3 weeks ago
- Tensor-Core selector unlock for RTX 4070 Ti Super☆29Oct 20, 2026Updated last week
- 基于 Stream 的 IPaaS 本地代理☆13Jan 2, 2025Updated last year
- ☆17Jun 3, 2026Updated 4 months ago
- Live dashboard: fire N parallel streaming coding-agent runs at any OpenAI-compatible endpoint — per-run TTFT/tok-s/E2E, real kill switch,…☆27Jul 23, 2026Updated 3 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Docker configuration for running VLLM on dual DGX Sparks☆2,468Updated this week
- Simulates a logged in user.☆16Jul 10, 2024Updated 2 years ago
- ☆16Apr 18, 2026Updated 6 months ago
- GlobalPoiner中文医疗文本嵌套与非嵌套医疗文本命名实体识别实验☆15May 7, 2022Updated 4 years ago
- High-performance multimodal OCR system built with LangChain 1.0 and vLLM. Integrates MinerU, PaddleOCR‑VL, and DeepSeek‑OCR with a unifie…☆15Nov 10, 2025Updated 11 months ago
- DeepSeek-V4.1-Flash-1017 on NVIDIA DGX Sparks with TensorFold + EXL3: TP4 (4 Sparks, 93.6 tok/s, 1M context) and TP2 (2 Sparks, 73 tok/s)…☆154Oct 26, 2026Updated last week
- Low-latency collective transport and distributed LLM serving stack for switchless GB10 clusters.☆102Updated this week
- Truthful scheduler-step and streaming visualization for local LLM inference☆35Sep 6, 2026Updated last month
- ☆20Sep 27, 2023Updated 3 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Educational reference: NVIDIA Blackwell SM100 vs SM120, NVFP4, tcgen05, MoE inference on consumer Blackwell☆29Sep 3, 2026Updated 2 months ago
- English guide + one-command patcher for the vLLM spin-wait bug that overheats NVIDIA DGX Spark / GB10. Credit: nacyot. CPU cores spin at …☆24Aug 17, 2026Updated 2 months ago
- DeepSeek V4 Flash @ 1M token context on 2x NVIDIA DGX Spark — production-tested recipe (45 tok/s decode, real 800K prompts served)☆37Jul 13, 2026Updated 3 months ago
- DeepSeek-v4-Flash 0731 recipe for 2x DGX Sparks☆1,507Oct 4, 2026Updated 3 weeks ago
- FlashQLA TileLang GDN kernels ported to NVIDIA Blackwell consumer (GB10 / DGX Spark)☆18Jun 5, 2026Updated 4 months ago
- ☆22Sep 5, 2020Updated 6 years ago
- Mixed-precision numerics benchmarks in Rust and Python - covering GEMMs, SYRKs, DOTs, and higher-level BLAS and LAPACK-style functionalit…☆33Updated this week