vLLM + Qwen3.6-27B (BF16) OpenAI-compatible inference server on AMD Strix Halo (Ryzen AI Max+ 395, gfx1151). Vision input, 256K context, /v1/responses with separated reasoning, via TheRock ROCm.
☆18Apr 26, 2026Updated 5 months ago
Alternatives and similar repositories for vllm-qwen
Users that are interested in vllm-qwen are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- vLLM Qwen 3.6-27B (AWQ-INT4) + DFlash speculative decoding on AMD Strix Halo (gfx1151 iGPU, 128 GB UMA, ROCm 7.13). 24.8 t/s single-strea…☆53May 10, 2026Updated 5 months ago
- Differential-KV (DKV) is a sparse KV-cache inference runtime designed for high-efficiency, memory-bounded long-context Large Language Mod…☆21Updated this week
- ☆23Updated this week
- WMMA GEMM in ROCm for RDNA GPUs☆19Sep 1, 2026Updated last month
- Evidence-backed AMD Strix Halo local-AI setup and benchmarks: Qwen3.8, Ollama, llama.cpp, Vulkan/ROCm, large GGUFs, and cross-OEM results…☆363Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 2x DGX Spark (GB10) running DeepSeek-V4-Flash: dual-node FP8 TP=2 (~41 tok/s single / ~350 agg), single-node ds4, a 10-model GGUF benchma…☆30Jun 16, 2026Updated 4 months ago
- Atlas Inference is a dependency free lightweight inference engine meant to run LLMs on YOUR compute seamlessly. Supports GB10 and Strix H…☆36Updated this week
- Persistent project-scoped knowledge base for Claude Code and Codex: quick and deep research depths, contrarian pass, source-independence …☆15Jul 23, 2026Updated 3 months ago
- Portable optimized ROCm and Qwen3.8-27B setup for AMD Strix Halo☆86Sep 16, 2026Updated last month
- Local AI setup for AMD Strix Halo APU - Lemonade + Vulkan + kyuz0☆87Updated this week
- Browser-based operator console for managing and monitoring Lemonade on Linux inference servers.☆21Aug 27, 2026Updated last month
- Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; port…☆218Oct 9, 2026Updated 2 weeks ago
- This is a mirror of the Strix Halo HomeLab wiki, to browse the wiki click on the link below☆74Aug 31, 2026Updated last month
- ☆29Jul 30, 2026Updated 2 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Windows GUI for controlled Bambu-compatible RFID and NFC tag workflows using Proxmark3.☆21Aug 19, 2026Updated 2 months ago
- A two-day, non-expert guide to running local LLMs on an AMD Strix Halo (Ryzen AI Max+ 395) box — full ~120GB memory pool, ROCm backend, N…☆31Jul 25, 2026Updated 2 months ago
- Container toolbox that runs llama.cpp with AMD ROCm and NVIDIA CUDA in one build, for inference on two GPUs☆20Oct 16, 2026Updated last week
- Raw speed benchmark traces: Qwen3.8-27B on 4x RTX 3090. llama.cpp GGUF sweeps + 288-point vLLM vs SGLang, NVFP4 vs FP8 matrix.☆13Sep 28, 2026Updated 3 weeks ago
- The fastest way to run Qwen3.8-Flash-Next on Strix Halo (gfx1151)☆1,084Updated this week
- A little project to convert Java Jar files into MacOS applications☆11Jun 29, 2017Updated 9 years ago
- LLM inference in C/C++☆42Updated this week
- SGLang Docker image for AMD Strix Halo (gfx1151 / Radeon 8060S)☆41Sep 17, 2026Updated last month
- Monitor temperature, control fan speeds, backlight color and more using WMI BIOS and the Embedded Controller. Lightweight, runs in the ba…☆15Oct 16, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- GPU + XDNA2 NPU inference for AMD Strix Halo (gfx1151): Qwen3.6-35B-A3B, Qwen3.8-27B and Qwen3.8 Flash-Next. NPU prefill +9% to +28%, MTP…☆75Updated this week
- Qwen3.8 Flash Next AWQ INT4 4x3090 v1: 262K, Vision, FP8 QSA KV and CUDA Graphs on 4x RTX 3090☆16Aug 30, 2026Updated last month
- Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.☆260Sep 7, 2026Updated last month
- ROCm/AMD GPU benchmark suite for llama.cpp, whisper.cpp, PyTorch☆21Feb 4, 2026Updated 8 months ago
- [ICLR 2026] Official Code of: "FlowCast: Advancing Precipitation Nowcasting with Conditional Flow Matching"☆29Aug 3, 2026Updated 2 months ago
- Ameoba Single key PCB variant designed for simplicity☆13Apr 17, 2023Updated 3 years ago
- Watch all Kubernetes Resources☆16Jun 16, 2026Updated 4 months ago
- Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux bu…☆58Updated this week
- ☆12Jul 3, 2020Updated 6 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Desync Recon & Exploitation☆11Nov 23, 2025Updated 11 months ago
- LLM Combinator exports real-time results of multi-modal Ollama executions with input from Frigate cameras and HA sensors into HomeAssista…☆14Nov 10, 2024Updated last year
- vLLM optimized for AMD☆27Updated this week
- Connect your Signoz Instance with Cursor, Claude Desktop or any other MCP Compatible Client☆14Mar 4, 2026Updated 7 months ago
- MCP server that shrinks logs, JSON, docs and web pages before your AI reads them. Offline, deterministic, ~3ms, no API key. 66% fewer tok…☆25Updated this week
- As name suggest it’s a filter which will take real world audio as input and remove all noises and produce clean phone conversation well. …☆11Jan 25, 2021Updated 5 years ago
- Random AI notes for working with local models or playing around with random machine learning bits.☆64Jun 7, 2026Updated 4 months ago