两张 V100 跑 Qwen3.8-27B 到 100 tok/s(186K 上下文):重写 sm70 注意力内核 + 多卡张量并行。Fork of geoffwatts/ninfer-v100.
☆21Sep 28, 2026Updated last month
Alternatives and similar repositories for ninfer-v100-tpx
Users that are interested in ninfer-v100-tpx are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- llama.cpp optimized for NVIDIA Tesla V100 (SM70). Qwen3.8-Flash-Next (MoE) with MTP on 1/2/3/6x V100-16GB at 262K context (6 GPUs: prefil…☆83Nov 12, 2026Updated last week
- High-performance Duo-V100-GPU inference for selected model checkpoints and GPUs.☆42Nov 2, 2026Updated 2 weeks ago
- ComfyUI MiniMax H3 inference optimizations for NVIDIA Tesla V100 (SM70)☆36Sep 29, 2026Updated last month
- A 198t/s and 353t/s qwen 3.8 27b inferencing engine for 2*v100 and 4*v100☆119Nov 12, 2026Updated last week
- Strata port for Tesla V100 (sm_70): Qwen3.8-Flash-Next with 262K context and MTP☆158Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/dis…☆66Updated this week
- NInfer on Tesla V100 (sm_70): Qwen3.8-27B deployment, build/convert scripts, systemd service, and cross-engine benchmark (vs llama.cpp). …☆56Sep 15, 2026Updated 2 months ago
- Lightweight Rust reimplementation of OpenClaw — self-hosted AI assistant, <10MB idle RAM, WASM sandboxed tools☆14Mar 1, 2026Updated 8 months ago
- Hermes plugin for safely editing Word, PowerPoint and Excel files: tracked changes, receipts, revert and a review Desk, built on paper-do…☆16Oct 15, 2026Updated last month
- The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; bench…☆126Updated this week
- Sol-Attn (arXiv 2607.24027) sparse acceleration for MiniMax H3 on V100 — single-node ComfyUI plugin (embedded FP16Safe + keep-or-drop ker…☆35Aug 29, 2026Updated 2 months ago
- Let an LLM write its own tools at runtime: parsed with acorn, versioned in SQLite, run in a supervised worker process☆24Sep 11, 2026Updated 2 months ago
- Fastest validated sglang recipe for Qwen3.8-27B (AWQ-INT4 + DSpark) on RTX 3090s — 1×/2×/4× GPU presets, image+video, full benchmarks☆66Oct 17, 2026Updated last month
- DeepSeek Harness React 视频创作工作台:内置官方 Remotion 技能、工程检索、模板库、素材排序、Studio 预览与本地 MP4 导出,支持原生清透 / 米白纸感 / Claude 暖书卷 / 8-Bit 像素 / 晴空薄荷五套明亮工作台主题。欢…☆40Updated this week
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Agent automation for IP-KVMs: one MCP server, CLI, fleet config, and skills driving GLKVM, PiKVM, and TinyPilot. Open-source alternative …☆24Aug 31, 2026Updated 2 months ago
- One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF…☆234Updated this week
- Strata for a PC with two NVIDIA GPUs and 32 GB of RAM: the low-RAM mode on a layer split, both cards working at the same time. 143 tok/s …☆85Oct 6, 2026Updated last month
- Typd: voice dictation that runs on your computer. One local speech model (llama.cpp) writes the finished text. Electron app for Windows a…☆22Nov 12, 2026Updated last week
- Skim pages, files, and lists with Jev: keep only what helps with your goal. Saves agent tokens. Built with go/cobra.☆33Oct 21, 2026Updated 3 weeks ago
- Run local LLMs on a Tesla V100-SXM2-16GB: prebuilt binaries, serve scripts, and setup for Windows and Linux (Qwen3 + Gemma 4, Claude Code…☆20Sep 1, 2026Updated 2 months ago
- Create markdown-backed Kanban boards in Obsidian.☆25Jul 2, 2024Updated 2 years ago
- 使用onnxruntime部署人群计数MPCount,包含C++和Python两个版本的程序☆10Aug 8, 2024Updated 2 years ago
- Simple Video Summarization using Text-to-Segment Anything (Florence2 + SAM2) This project provides a video processing tool that utilizes…☆10Feb 20, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- MiniMax-H3 Fun ControlNet for ComfyUI. Depth, canny, pose, HED or MLSD video driving H3 frame by frame, built on ComfyUI's own H3 blocks.☆53Sep 15, 2026Updated 2 months ago
- xllama.cpp — Optimized LLM Inference for NVidia Tesla V100-SXM2-32GB☆18Apr 29, 2026Updated 6 months ago
- A multimodal large-scale model, which performs close to the closed-source Qwen-VL-PLUS on many datasets and significantly surpasses the p…☆14Feb 5, 2024Updated 2 years ago
- Slidev implementation☆19Jul 16, 2026Updated 4 months ago
- Cog wrapper for moondream2☆14Oct 31, 2026Updated 2 weeks ago
- DirectorX as a DeepSeek Harness plugin: AI video/image/audio skills, knowledge corpus, and configurable vision/image/video/audio model to…☆51Oct 9, 2026Updated last month
- Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~4…☆542Updated this week
- 保密型聊天软件服务器 /Private Chat Server☆17May 15, 2016Updated 10 years ago
- Submodule for Grounded-SAM☆12Apr 17, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- OpenAI-compatible inference server for 4-bit Qwen3.8-27B on one RTX 3090, 262K context☆34Updated this week
- 智联云城市水位监控系统(硬件+前后端+AI接入)☆14Sep 22, 2022Updated 4 years ago
- 使用OpenCV+onnxruntime部署开放域目标检测,包含C++和Python两个版本的程序☆11Jan 4, 2024Updated 2 years ago
- Simple CogVLM client script☆13Dec 20, 2023Updated 2 years ago
- OpenRouter for agent security.☆71Nov 4, 2026Updated 2 weeks ago
- Real-Time Open-Vocabulary Object Detection☆12Feb 7, 2024Updated 2 years ago
- run a 27b local ai on your gaming gpu, rtx 3060 and up, radeon rx 6600 and up in beta, 8gb or more, linux and windows, one line to instal…☆192Oct 17, 2026Updated last month
Are these results useful?