LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.
☆474Sep 22, 2026Updated last month
Alternatives and similar repositories for Lvllm
Users that are interested in Lvllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆164Updated this week
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated 11 months ago
- 大语言模型工具集☆28Aug 1, 2025Updated last year
- The complete NUMA-optimized branch of the ktransformers project☆27Nov 3, 2025Updated 11 months ago
- A fork of jasl/vllm (codex/ds4-sm120-min-enable) with CPU-GPU hybrid inference support for DeepSeek-V4 on SM120+.☆32Sep 10, 2026Updated last month
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆61Sep 16, 2026Updated last month
- fastllm是后端无依赖的高性能大模型推理库，帮助您在大部分设备上高效推理大模型！☆5,130Updated this week
- forked from vllm-project/flash-attention☆70May 9, 2026Updated 5 months ago
- Local LLM Inference Speed Test Tool☆247Oct 21, 2026Updated last week
- The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-reques…☆1,194Oct 1, 2026Updated last month
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,431Updated this week
- DeepSeek-V4.1-Flash (764B) on 4×CMP 170HX: PP4 + shadow KV + EXL3 2bpw + DSpark + 512k ctx + native vision — patches, recipes, acceptance…☆15Sep 29, 2026Updated last month
- Unlock P2P comms between consumer NVIDIA GPUs☆684Updated this week
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆198Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations☆19,647Updated this week
- Cross-tier CPU/GPU hybrid MoE inference for SGLang (plugin, tracks upstream main)☆47Updated this week
- llama.cpp fork with additional SOTA quants and improved performance☆3,354Updated this week
- A tool to unlobotomize your NVIDIA card!☆818Updated this week
- high-performance linear attention kernel library built on TileLang☆14Jun 8, 2026Updated 4 months ago
- DeepSeek-V4-Flash W4A16-FP8 quantization on 8x H200 — patches, recipe, mission report☆14Sep 26, 2026Updated last month
- A high-throughput and memory-efficient inference and serving engine for LLMs☆38Updated this week
- run DeepSeek-R1 GGUFs on KTransformers☆258Mar 3, 2025Updated last year
- vLLM 0.32 fork for consumer GPUs: 3 x 262K sessions on 4x RTX 3090, Qwen 3.8 Flash-Next at 184 tok/s, up to 17 on bigger cards. Seven che…☆31Oct 22, 2026Updated last week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations☆45Sep 10, 2026Updated last month
- vLLM for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60☆436Feb 20, 2026Updated 8 months ago
- SGLang is a fast serving framework for large language models and vision language models.☆16Updated this week
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区，聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小 构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆26Jul 2, 2026Updated 3 months ago
- NVIDIA Linux open GPU with P2P support☆42Updated this week
- Dogfooding vLLM backport for older GPUs.☆345Updated this week
- a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-…☆474Updated this week
- Single-request LLM serving recipes, patches, and benchmarks for modified RTX 2080 Ti 22GB / SM75 systems☆97Jun 10, 2026Updated 4 months ago
- A ComfyUI custom node enabling **Flash Attention 1** on legacy NVIDIA GPUs (Tesla V100, T4) that lack Compute Capability 8.0+ required by…☆33Feb 9, 2026Updated 8 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,223Updated this week
- The most comprehensive community resource for the NVIDIA CMP 170HX.☆130Jul 30, 2026Updated 3 months ago
- Qwen3.8-Flash-Next (W4A16) on 2x CMP 170HX: 110 tok/s decode, 7K prefill, 800 tok/s @ 23 concurrent - runs on 38 GB home RAM, no pinned m…☆20Sep 18, 2026Updated last month
- 限制问题区域显存 62906MiB / 65024MiB 支持驱动 610.57.04 ，修改显卡显示名称为NVIDIA Tesla A100☆13Aug 30, 2026Updated 2 months ago
- ☆43May 6, 2025Updated last year
- ☆25Oct 20, 2026Updated last week
- KTransformers 一键部署脚本☆60Apr 18, 2025Updated last year