vLLM 0.33 fork for consumer GPUs: 3 x 262K sessions on Ampere,Ada and more! Qwen 3.8 Flash-Next at 207 tok/s, up to 17 on bigger cards. Seven checkpoints incl. Qwen3.8-27B, Nemotron 3.5 Super VL and Ornith 1.5. One command: merlin serve.
☆50Nov 12, 2026Updated last week
Alternatives and similar repositories for merlin-engine
Users that are interested in merlin-engine are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Qwen3.8 Flash Next AWQ INT4 4x3090 v1: 262K, Vision, FP8 QSA KV and CUDA Graphs on 4x RTX 3090☆16Aug 30, 2026Updated 2 months ago
- Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,86…☆145Oct 15, 2026Updated last month
- some experiments about 170hx 10g version.☆14Oct 19, 2026Updated last month
- What I run on my inference server: Qwen3.8-Flash-Next on two RTX 3090s, and how I got here☆18Oct 12, 2026Updated last month
- Local-first, LLM-agnostic memory system — hybrid retrieval (vector + lexical + graph), multi-signal ranking. Runs on 4GB RAM, CPU-only, n…☆17Oct 8, 2026Updated last month
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Qwen3.8-Flash-Next (W4A16) on 2x CMP 170HX: 110 tok/s decode, 7K prefill, 800 tok/s @ 23 concurrent - runs on 38 GB home RAM, no pinned m…☆20Sep 18, 2026Updated 2 months ago
- 双 SM80 / CMP 170HX 的 Qwen3.8-Flash-Next 社区 vLLM:NVIDIA NVFP4、GDS、TEP2、MTP6,支持 1~10 并发,词表与 PCIe 通信优化。中英文部署与可构建源码。☆28Oct 15, 2026Updated last month
- 🚀 Neovim plugin to open API endpoints directly in the editor ⚡☆12Dec 21, 2023Updated 2 years ago
- A cmpunlocker fork with working p2p on SYS-4028GR-TRT2☆27Aug 8, 2026Updated 3 months ago
- NInfer SM80 port for NVIDIA CMP 170HX☆29Aug 22, 2026Updated 2 months ago
- Qwen3.8-27B INT8 + DFlash2 on a rented CMP 170HX — failure analysis (private GHCR image prevented benchmark)☆17Aug 30, 2026Updated 2 months ago
- Open quantization tooling for TurboQuant-style low-bit LLM releases, stock GGUF deployment, and Apple Silicon runtime experiments.☆10May 5, 2026Updated 6 months ago
- Fastest validated sglang recipe for Qwen3.8-27B (AWQ-INT4 + DSpark) on RTX 3090s — 1×/2×/4× GPU presets, image+video, full benchmarks☆66Oct 17, 2026Updated last month
- defeating fuse security on ga100 with pico 2(and some wires)☆112Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- OpenAI-compatible inference server for 4-bit Qwen3.8-27B on one RTX 3090, 262K context☆34Updated this week
- Ansible All The Things!☆14Apr 30, 2026Updated 6 months ago
- Misc personal tools that don't deserve their own repo yet☆14Oct 23, 2026Updated 3 weeks ago
- Unlock P2P comms between consumer NVIDIA GPUs☆735Oct 26, 2026Updated 3 weeks ago
- Raw benchmark logs for Swift-Qwen3.8-27B vs the Qwen3.8-27B BF16 base: every response, score, and config across 9 benchmarks☆13Sep 13, 2026Updated 2 months ago
- Multi-GPU (tensor-parallel) CUDA megakernel for Qwen3.8-27B GGUF (Q4_K_M, Q5_K_M, Q6_K, Q8_0) with on-device MTP speculative decoding. Fo…☆16Oct 23, 2026Updated 3 weeks ago
- Not insane. Designed to read WDB files from the latest World of Warcraft builds and output CSV files. Could be used as a library by other…☆15Sep 27, 2025Updated last year
- Fast open-source (MIT) LLM inference engine in C++/CUDA for RTX 3090-class GPUs: Qwen3.8-Flash-Next (~1,000 tok/s with 8 agents on 3x RTX…☆71Updated this week
- Terminal benchmark dashboard for LLM inference servers (OpenAI-compatible): live prefill/decode throughput, latency percentiles, concurre…☆14Sep 5, 2026Updated 2 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆18Aug 26, 2025Updated last year
- Serving Qwen3.8-27B at ~140 tok/s on 2x AMD R9700 in a Proxmox VM — with working GPU P2P (emulated PCIe switch + XanMod kernel + NDEBUG R…☆21Sep 16, 2026Updated 2 months ago
- Serve EXL3 (ExLlamaV3 trellis) quantized models on vLLM fork runtimes — any architecture, mixed per-layer bitrates, composable with sourc…☆27Nov 5, 2026Updated 2 weeks ago
- See what Hermes Agent left on your disk, inside Hermes Desktop: a treemap by what each folder is to Hermes, git-aware cards for what can …☆19Oct 23, 2026Updated 3 weeks ago
- 公安一网通办App(by 中华人民共和国公安部科技信息化局),安全审计报告发布用☆16Oct 25, 2026Updated 3 weeks ago
- llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ to…☆398Updated this week
- Give text-only LLMs vision. A tiny OpenAI-compatible proxy that lets reasoning models (DeepSeek, Qwen, GLM…) see images by querying a sep…☆49Jul 7, 2026Updated 4 months ago
- Dragonflight_Dekk-Core☆20Dec 23, 2023Updated 2 years ago
- J-tau: A Fully Localized Japanese Benchmark for Tool-Agent-User Interaction☆18Oct 15, 2026Updated last month
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- DsSScan — is a tool for scanning directories contained in .DS_Store files used on websites. It extracts information about the directory s…☆10Jan 27, 2025Updated last year
- Qwen3.8-27B QSA for llama.cpp with MTP, native multimodal attention, and prefix-cache reuse☆16Nov 5, 2026Updated 2 weeks ago
- Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when …☆2,751Nov 10, 2026Updated last week
- Bash script presents crontab in human readable form.☆11Apr 21, 2018Updated 8 years ago
- ☆13Mar 26, 2026Updated 7 months ago
- Real-time AI dictation using faster-whisper—type anywhere with instant, accurate speech-to-text conversion.☆11Mar 25, 2025Updated last year
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆15Sep 29, 2026Updated last month
Are these results useful?