Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
☆2,580Nov 3, 2026Updated this week
Alternatives and similar repositories for HyperQwen
Users that are interested in HyperQwen are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux bu…☆502Updated this week
- Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and m…☆2,477Updated this week
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,269Updated this week
- Run GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- …☆2,585Updated this week
- llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ to…☆223Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆40,925Updated this week
- Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,86…☆134Oct 15, 2026Updated 3 weeks ago
- Your GeForce runs AI at half speed. This fixes it for 20/30/40 series cards. Up to 2× FP16 / BF16 / TF32 tensor throughput☆17,323Oct 29, 2026Updated last week
- LLM speculative inference server for heterogeneous hardware & consumer GPUs☆2,976Updated this week
- Deployment kit: Qwen3.8-27B EXL3 3.5bpw target + DFlash2 EXL3 5.0bpw speculative draft — launcher, config, model cards☆233Sep 1, 2026Updated 2 months ago
- Xolotl (formerly Strata-DSV4.1): run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 90 GB RAM) at up…☆467Updated this week
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,183Updated this week
- Unlock P2P comms between consumer NVIDIA GPUs☆697Oct 26, 2026Updated last week
- Run Qwen3.8-27B at 140 tok/s on a single RTX 3090, up to 2x llama.cpp. Fast local LLM inference server with OpenAI- and Anthropic-compati…☆72Updated this week
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Fastest validated sglang recipe for Qwen3.8-27B (AWQ-INT4 + DSpark) on RTX 3090s — 1×/2×/4× GPU presets, image+video, full benchmarks☆64Oct 17, 2026Updated 3 weeks ago
- Source-available C++/CUDA inference engine for Qwen3.8-27B INT4 on 3x RTX 3090☆41Updated this week
- GLM-5.3-Flash EXL3 on one 24 GB RTX 3090 + DDR4: elastic GPU expert cache, zero-copy experts, AVX2 CPU tier, OpenAI API☆349Oct 18, 2026Updated 2 weeks ago
- Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell☆694Updated this week
- GVS5H: Five Qwen3.8-27B Models Match Claude Fable 5 on LiveCodeBench Hard | Fable 5 Level Coding for a Fifth the Price - or on a Single…☆1,198Updated this week
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,794Updated this week
- One llama.cpp flag unlocks +33-39% decode speed for Qwen3.8-27B on consumer GPUs. The MTP head already ships inside your GGUF. Recipe, pa…☆367Sep 26, 2026Updated last month
- llama.cpp fork with additional SOTA quants and improved performance☆3,377Updated this week
- Code for Latent Sparse Attention: Towards Subquadratic Ultra-Long Context Attention☆421Updated this week
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- DeepSeek-V4.1-Flash-1017 (552B MoE + 196B Engram) on 2x RTX 3090 + 128 GB RAM. Rust + CUDA, T16 quantization. 40+ tok/s measured with exp…☆91Updated this week
- Production Docker deployment for Qwen3.8-27B AutoRound W4A16 (Vishva007) on 2x RTX 3090 (NVLink or PCIe). OpenAI-compatible vLLM with MTP…☆19Aug 20, 2026Updated 2 months ago
- Experimental llama.cpp fork for inference research and development☆900Updated this week
- Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don…☆6,180Updated this week
- ☆30Updated this week
- An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, a…☆4,077Updated this week
- Qwen3.8-27B on 16-32 GB Nvidia GPUs one-click install for Windows / Linux☆600Oct 9, 2026Updated 3 weeks ago
- C++/CUDA single-GPU inference engine for Qwen3.8 (incl. Qwen3.8-Flash-Next with offloaded experts) on the RTX 5090; grew from NInfer☆275Updated this week
- Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith …☆887Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Give GeForce cards (Turing and later) a BAR1 at least as large as their VRAM on Linux, without flashing.☆21Updated this week
- One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF…☆196Updated this week
- Mirai S (Qwen3.8-27B-S trellis GGUF) on a 12 GB RTX 4070: 262k q8_0 tiered cache, MTP drafting at every depth, harness-proofing, the laye…☆109Oct 26, 2026Updated last week
- A tool to unlobotomize your NVIDIA card!☆834Oct 28, 2026Updated last week
- LLM inference in C/C++☆2,444Oct 22, 2026Updated 2 weeks ago
- DeepSeek-V4.1-Flash EXL3 on one 24 GB RTX 3090 + DDR4 + NVMe: staged-DMA prefill, AVX2 CPU expert tier, elastic VRAM expert cache, Engram…☆155Oct 12, 2026Updated 3 weeks ago
- An offline coding agent for local models. One command sets up the best model your GPU can run, and on 24 GB+ RTX 30/40/50-series cards Qw…☆136Updated this week