CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.
☆61Sep 16, 2026Updated last month
Alternatives and similar repositories for Lvllmds4-x
Users that are interested in Lvllmds4-x are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆166Updated this week
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆474Updated this week
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆203Updated this week
- The complete NUMA-optimized branch of the ktransformers project☆27Nov 3, 2025Updated last year
- Personal fork of Kacper’s vLLM-Moet for DeepSeek V4 Flash on a single RTX PRO 6000. Shared for others to use; support is best-effort as t…☆18Aug 19, 2026Updated 2 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A Pytorch Extention for CMP 170HX☆17Jun 29, 2026Updated 4 months ago
- 1Cat-vLLM on Intel Gaudi☆47Oct 12, 2026Updated 3 weeks ago
- ☆11Oct 18, 2026Updated 3 weeks ago
- GLM-5.3-Flash (320B MoE) at TP4 across four DGX Sparks, same day as the model drop: 36 tok/s, 1.26M-token FP8 KV pool, 262K context, MTP …☆57Oct 10, 2026Updated 3 weeks ago
- DeepSeek-V4.1-Flash-1017 (552B MoE + 196B Engram) on 2x RTX 3090 + 128 GB RAM. Rust + CUDA, T16 quantization. 40+ tok/s measured with exp…☆93Updated this week
- vLLM fork for pascal gpu☆18Mar 13, 2026Updated 7 months ago
- RocketLLama. Fork off LLama cpp that newly scales 2-20x now also on big models not fully fitting in vram on even just 2 gpus☆16Updated this week
- Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~4…☆479Updated this week
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated last year
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Metal acceleration project for the ASRock BC-250 on macOS. Based on ChefKiss' NootedRed project. LLMs are used in this project for debugg…☆39Oct 1, 2026Updated last month
- DeepSeek-V4.1-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 297 tok/s for one user, up to 868 tok/s at eight, 262K context, one-command c…☆23Updated this week
- Proxy for OpenAI☆16Sep 2, 2025Updated last year
- LLM inference in C/C++☆75Updated this week
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆15Sep 29, 2026Updated last month
- SonicScribe 是一个基于 GLM-ASR-Nano-2512 模型的 Web 原生实时语音转文字系统，采用前后端分离架构，支持低延迟、高准确率的语音识别服务。适用于会议记录、讲座整理、访谈笔记、日常对话等多种场景。☆25Jan 12, 2026Updated 9 months ago
- Self-hosted game stream host for Moonlight.☆17Jul 4, 2026Updated 4 months ago
- Builds vLLM for ROCm gfx906 using Nix☆18Aug 29, 2026Updated 2 months ago
- glm-5.2 on too many 64 / 80 GB A100's☆22Sep 9, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Async PLE SSD offload for Qwen3.8 Flash Next on a 64 GiB SM80 GPU: vLLM patch, setup guide, and measured PP/TG results.☆23Sep 24, 2026Updated last month
- A proxy that hosts multiple single-model runners such as LLama.cpp and vLLM☆12May 30, 2025Updated last year
- ☆28Oct 7, 2026Updated last month
- Fully arbitrary r/w and code execution on the BC-250 SMU☆13Sep 18, 2026Updated last month
- ☆11Nov 10, 2025Updated 11 months ago
- 大语言模型工具集☆28Aug 1, 2025Updated last year
- AGX Dynamics for Unreal plugin.☆13Updated this week
- forked from vllm-project/flash-attention☆74May 9, 2026Updated 5 months ago
- fastllm是后端无依赖的高性能大模型推理库，帮助您在大部分设备上高效推理大模型！☆5,131Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- HX 家族多功能工具箱：NVIDIA CMP 全系矿卡（30HX/40HX/50HX/70HX/90HX/170HX）工作台——固件解锁、型号伪装、双卡调度、动态省电、快照回滚。单文件免安装，前端与工具包全部内嵌。☆41Oct 25, 2026Updated 2 weeks ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆40Aug 13, 2026Updated 2 months ago
- AI agent with CLI, API and all prompts editable.☆16Updated this week
- autonomous overnight code quality bot for Hermes Agent☆14May 28, 2026Updated 5 months ago
- Framebot is a C framework for Telegram Bot API.☆15Mar 15, 2019Updated 7 years ago
- Free and Open-Source TailwindCSS styled components designed for integration with shadcn, v0 and React/Next.js projects.☆12Jun 29, 2026Updated 4 months ago
- ☆30May 12, 2026Updated 5 months ago