LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.
☆476Nov 6, 2026Updated 2 weeks ago
Alternatives and similar repositories for Lvllm
Users that are interested in Lvllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆169Updated this week
- vllm混合推理扩展插件,支持多NUMA混合推理,单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated last year
- 大语言模型工具集☆28Aug 1, 2025Updated last year
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆61Sep 16, 2026Updated 2 months ago
- The complete NUMA-optimized branch of the ktransformers project☆27Nov 3, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- fastllm是后端无依赖的高性能大模型推理库,帮助您在大部分设备上高效推理大模型!☆5,133Updated this week
- Dogfooding vLLM backport for older GPUs.☆14Oct 29, 2026Updated 3 weeks ago
- forked from vllm-project/flash-attention☆75May 9, 2026Updated 6 months ago
- Cross-tier CPU/GPU hybrid MoE inference for SGLang (plugin, tracks upstream main)☆85Updated this week
- Local LLM Inference Speed Test Tool☆256Oct 21, 2026Updated last month
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆209Updated this week
- Unlock P2P comms between consumer NVIDIA GPUs☆744Oct 26, 2026Updated 3 weeks ago
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,550Updated this week
- A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations☆19,686Nov 13, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- dsv41-flash-pp5-170hx &pp6 Supports the original (full/unquantized) model. Prefill 7,000 t/s, decode 45-75-125 t/s. dspark. Requires 200 …☆13Updated this week
- The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-reques…☆1,236Oct 1, 2026Updated last month
- 1Cat-vLLM on Intel Gaudi☆48Oct 12, 2026Updated last month
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆15Sep 29, 2026Updated last month
- DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)☆36Jul 2, 2026Updated 4 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆3,408Updated this week
- SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35…☆133Sep 30, 2026Updated last month
- ☆43May 6, 2025Updated last year
- Dogfooding vLLM backport for older GPUs.☆365Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- run DeepSeek-R1 GGUFs on KTransformers☆257Mar 3, 2025Updated last year
- vLLM for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60☆436Feb 20, 2026Updated 9 months ago
- ☆11Updated this week
- DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallel…☆120Aug 13, 2026Updated 3 months ago
- A tool to unlobotomize your NVIDIA card!☆940Oct 28, 2026Updated 3 weeks ago
- high-performance linear attention kernel library built on TileLang☆15Jun 8, 2026Updated 5 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆38Nov 4, 2026Updated 2 weeks ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,387Updated this week
- a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-…☆489Updated this week
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区,聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆27Jul 2, 2026Updated 4 months ago
- defeating fuse security on ga100 with pico 2(and some wires)☆141Updated this week
- 限制问题区域显存 62906MiB / 65024MiB 支持驱动 610.57.04 ,修改显卡显示名称为NVIDIA Tesla A100☆12Aug 30, 2026Updated 2 months ago
- vLLM for NVIDIA CMP 170HX (Ampere sm_80): fast, repeatable model serving on one card or several☆32Nov 2, 2026Updated 2 weeks ago
- Qwen3.8-Flash-Next (W4A16) on 2x NVIDIA CMP 170HX with vLLM pipeline parallel: 226 tok/s single-user, 840 tok/s at 8 users, 262K context,…☆12Nov 11, 2026Updated last week
- Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B) and Hy4-Flash (220B) on your own GPUs - f…☆3,106Updated this week
- Container toolbox that runs llama.cpp with AMD ROCm and NVIDIA CUDA in one build, for inference on two GPUs☆21Oct 16, 2026Updated last month
Are these results useful?