vLLM Qwen 3.6-27B (AWQ-INT4) + DFlash speculative decoding on AMD Strix Halo (gfx1151 iGPU, 128 GB UMA, ROCm 7.13). 24.8 t/s single-stream, vision, tool calling, 256K context, OpenAI-compatible, Docker. Matches DGX Spark FP8+DFlash+MTP at a third of the cost. No CUDA.
☆53May 10, 2026Updated 6 months ago
Alternatives and similar repositories for vllm-awq4-qwen
Users that are interested in vllm-awq4-qwen are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- vLLM + Qwen3.6-27B (BF16) OpenAI-compatible inference server on AMD Strix Halo (Ryzen AI Max+ 395, gfx1151). Vision input, 256K context, …☆19Apr 26, 2026Updated 6 months ago
- SGLang Docker image for AMD Strix Halo (gfx1151 / Radeon 8060S)☆44Sep 17, 2026Updated last month
- Home-enthusiast's guide to fine-tuning 27B+ LLMs on AMD Strix Halo (gfx1151, Ryzen AI MAX+ 395) — the patches and tuning to make Linux ma…☆32Jun 9, 2026Updated 5 months ago
- Docker Compose for llama.cpp GGUF servers on AMD Strix Halo: Qwen, Gemma, and Laguna packages (abliterated and quantized), stock Vulkan p…☆28Aug 17, 2026Updated 2 months ago
- Persistent project-scoped knowledge base for Claude Code and Codex: quick and deep research depths, contrarian pass, source-independence …☆15Jul 23, 2026Updated 3 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Portable vLLM builds with AMD ROCm acceleration for Lemonade☆81Sep 9, 2026Updated 2 months ago
- ☆549Nov 2, 2026Updated last week
- Evidence-backed AMD Strix Halo local-AI setup and benchmarks: Qwen3.8, Ollama, llama.cpp, Vulkan/ROCm, large GGUFs, and cross-OEM results…☆373Updated this week
- LLM inference in C/C++☆43Nov 5, 2026Updated last week
- Container toolbox that runs llama.cpp with AMD ROCm and NVIDIA CUDA in one build, for inference on two GPUs☆22Oct 16, 2026Updated 3 weeks ago
- ☆18Nov 4, 2026Updated last week
- Self-hosted, browser-based AI dungeon RPG, fully local on an AMD Strix Halo APU: narrator + per-character agents on one local LLM (Gemma …☆30Jul 25, 2026Updated 3 months ago
- NPU-accelerated speech transcription & speaker diarization on AMD Strix Halo: Whisper-v3-turbo on XDNA 2 + pyannote, OpenAI-compatible AP…☆14Aug 31, 2026Updated 2 months ago
- Experimental support for many TTS/STT LLMs wrapped in a Wyoming API for consumption via Homeassistant☆45Sep 13, 2026Updated last month
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- A Deepseek V4 Flash 0731 / Vision Exp VLLM stack☆27Sep 24, 2026Updated last month
- Exact LLM inference engine written from scratch for AMD Strix Halo (gfx1151, ROCm/HIP). Qwen3.8-Flash-Next on one node or two replicas; D…☆21Updated this week
- Rust LLM inference engine tuned for AMD Strix Halo (gfx1151): a fork of Strata with ggml/llama.cpp-derived HIP kernels. Five models on on…☆40Updated this week
- Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; port…☆225Oct 25, 2026Updated 2 weeks ago
- Open-source self-hosted home AI inference platform for AMD Strix Halo — multi-backend slots, OpenAI-compatible gateway, Vue 3 + FastAPI +…☆85Updated this week
- Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthrop…☆1,300Updated this week
- ☆180Sep 2, 2026Updated 2 months ago
- llama.cpp setup on dedicated AMD Strix Halo machine☆14Jul 26, 2026Updated 3 months ago
- LLM Fine Tuning Toolbox images for Ryzen AI 395+ Strix Halo☆66Sep 12, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆2,024Updated this week
- experimental linux kernel modules and userspace drivers for RDMA-over-usb4/thunderbolt on consumer hardware☆75Oct 19, 2026Updated 3 weeks ago
- ☆20Sep 4, 2025Updated last year
- CIRU llama.cpp runtime for Qwen3.8-Flash-CIRU-STRIX-IU4 on AMD Strix Halo☆22Updated this week
- RocM Optimized ComfyUI nodes☆52Oct 25, 2026Updated 2 weeks ago
- Dockernized ComfyUI with PyTorch & flash-attention for gfx1151 (AMD Strix Halo, Ryzen AI Max+ 395), relying on AMD's pre-built and pre-co…☆45Feb 25, 2026Updated 8 months ago
- Lemonade Pi Agent CLI Plugin☆17Sep 12, 2026Updated 2 months ago
- The Open Source Strix Halo AI engine. Qwen 3.8 Flash Next Q4_K_XL: 1,801.01 pp, 73.17 tg single user, 194.37 tok/s 8 users; Qwen 3.8 27B…☆1,142Updated this week
- Two-rank tensor-parallel LLM serving over USB4v2 between AMD Strix Halo hosts: a custom Linux thunderbolt stream transport, self-healing …☆17Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Serve GLM-5.2 469B (REAP-pruned, NVFP4) across 3× NVIDIA DGX Spark with vLLM pipeline parallelism — 256K context, production-ready config…☆34Jul 12, 2026Updated 4 months ago
- Experimental runtime reclamation of GB10 display-reserved memory into ordinary Linux RAM, with guarded driver, local signing, and reboot …☆13Oct 23, 2026Updated 2 weeks ago
- A deterministic inference engine for DeepSeek-V4.1-Flash 1017 and Qwen3.8-Flash-Next on a single AMD Strix Halo 128 GB☆21Updated this week
- GPU + XDNA2 NPU inference for AMD Strix Halo (gfx1151): Qwen3.6-35B-A3B, Qwen3.8-27B, Qwen3.8 Flash-Next, Flint and GLM-5.3 Flash (321B M…☆94Updated this week
- Qwen3.8-27B EXL3 + DFlash2 speculative decoding on an AMD RX 7900 XTX (exllamav3-rocm): recipe, patches, benchmarks☆22Nov 2, 2026Updated last week
- ☆29Jun 10, 2026Updated 5 months ago
- vLLM inference solutions for NVIDIA GB10 (DGX Spark, sm121) — KIMI-K3 B12X_MLA + DSpark, GLM-5.2☆69Sep 30, 2026Updated last month
Are these results useful?