DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)
☆35Jul 2, 2026Updated 3 months ago
Alternatives and similar repositories for vllm-dsv4-ampere
Users that are interested in vllm-dsv4-ampere are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Dogfooding vLLM backport for older GPUs.☆345Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆38Updated this week
- Single-request LLM serving recipes, patches, and benchmarks for modified RTX 2080 Ti 22GB / SM75 systems☆97Jun 10, 2026Updated 4 months ago
- An extension to oobabooga's TextGen allowing you to receive pics generated by Automatic1111's SD API☆11May 16, 2023Updated 3 years ago
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated 11 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- ☆25Oct 20, 2026Updated last week
- high-performance linear attention kernel library built on TileLang☆14Jun 8, 2026Updated 4 months ago
- vLLM for NVIDIA CMP 170HX (Ampere sm_80): fast, repeatable model serving on one card or several☆23Updated this week
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆198Updated this week
- Generates kid-friendly weather forecasts suitable for display on a large monitor. Uses OpenWeatherMap and a local or hosted LLM.☆17Feb 28, 2026Updated 8 months ago
- Optimized inference for the DGX Spark and spark clusters☆131Updated this week
- Qwen3.8-Flash-Next (W4A16) on 2x CMP 170HX: 110 tok/s decode, 7K prefill, 800 tok/s @ 23 concurrent - runs on 38 GB home RAM, no pinned m…☆20Sep 18, 2026Updated last month
- ☆14Apr 8, 2025Updated last year
- 双 SM80 / CMP 170HX 的 Qwen3.8-Flash-Next 社区 vLLM：NVIDIA NVFP4、GDS、TEP2、MTP6，支持 1～10 并发，词表与 PCIe 通信优化。中英文部署与可构建源码。☆28Oct 15, 2026Updated 2 weeks ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆474Sep 22, 2026Updated last month
- eBPF Tools - Tool for monitoring, performance benchmarking and tracing linux kernel☆17Jan 29, 2021Updated 5 years ago
- [NAACL'24] ZGLS: Zero-shot Generative Linguistic Steganography☆13Jan 30, 2025Updated last year
- ☆12Jan 22, 2026Updated 9 months ago
- Exact SGLang runtime for GLM-5.3-Flash-NVFP4 on 4x RTX PRO 6000 Blackwell (SM120): flashinfer_cutlass MoE, DSA+KDA attention, NEXTN MTP5,…☆17Aug 27, 2026Updated 2 months ago
- ☆15Updated this week
- Official code for paper "Revisiting Model Interpolation for Efficient Reasoning"☆18Jul 14, 2026Updated 3 months ago
- A powerful Model Context Protocol (MCP) server that converts Markdown files and content to beautifully styled PDFs with Mermaid diagram s…☆11Jul 17, 2025Updated last year
- Code for our paper: "Building A Coding Assistant via Retrieval-Augmented Language Models"☆10Nov 2, 2024Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Production Docker deployment for Qwen3.8-27B AutoRound W4A16 (Vishva007) on 2x RTX 3090 (NVLink or PCIe). OpenAI-compatible vLLM with MTP…☆18Aug 20, 2026Updated 2 months ago
- Tutorial for Writing a Kernel☆10Jun 5, 2022Updated 4 years ago
- Simple Artificial Neural Network library implemented in Rust.☆12Dec 6, 2022Updated 3 years ago
- Clean-room AGPL-3.0 reimplementation of the halogen 0.1.3 inference engine for Qwen3.8-27B on AMD Strix Halo (gfx1151) — same wire protoc…☆18Sep 10, 2026Updated last month
- Give the DeepSeek Harness (dsh) eyes with any text model + any vision model: a vision proxy for chat image attachments and an analyze_ima…☆12Sep 28, 2026Updated last month
- ALPS: An Adaptive Learning, Priority OS Scheduler for Serverless Functions (USENIX ATC'24)☆15Jun 20, 2024Updated 2 years ago
- Agent-ready two-node DGX Spark recipe for DeepSeek V4 Flash plus an optional Qwen Vision sidecar, with generic OpenAI-compatible, Hermes,…☆24Aug 10, 2026Updated 2 months ago
- Fast and efficient attention method exploration and implementation.☆29Jul 6, 2026Updated 3 months ago
- NVIDIA Fan Control Service tested on Ubuntu providing more aggressive fan curves for RTX Pro 6000 GPUs☆19Jan 17, 2026Updated 9 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆11Mar 1, 2023Updated 3 years ago
- FPGA Development toolset☆20Jun 15, 2017Updated 9 years ago
- Animation Engine | 动画引擎 | Hi Animation | Hello Animation | WIP☆18Sep 13, 2024Updated 2 years ago
- Ressys benchmark code repo☆12Dec 5, 2025Updated 10 months ago
- 一个Turing专用vLLM分支，通过整合FlashInfer 0.6.18（PR #3526）与SM75后端门槛补丁（PR #47949），持续同步vLLM主线最新版本☆37Oct 11, 2026Updated 2 weeks ago
- Code and Data for PeerQA: A Scientific Question Answering Dataset from Peer Reviews, NAACL 2025 https://aclanthology.org/2025.naacl-long.…☆17Jun 1, 2026Updated 5 months ago
- Run local LLMs on a Tesla V100-SXM2-16GB: prebuilt binaries, serve scripts, and setup for Windows and Linux (Qwen3 + Gemma 4, Claude Code…☆19Sep 1, 2026Updated 2 months ago