Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parallel architecture, suitable for MOE model hybrid inference.
☆168Nov 11, 2026Updated this week
Alternatives and similar repositories for Lsglang
Users that are interested in Lsglang are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆474Nov 6, 2026Updated last week
- A fork of jasl/vllm (codex/ds4-sm120-min-enable) with CPU-GPU hybrid inference support for DeepSeek-V4 on SM120+.☆32Sep 10, 2026Updated 2 months ago
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆61Sep 16, 2026Updated last month
- 大语言模型工具集☆28Aug 1, 2025Updated last year
- ☆11Oct 18, 2026Updated 3 weeks ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Cross-tier CPU/GPU hybrid MoE inference for SGLang (plugin, tracks upstream main)☆72Updated this week
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated last year
- The complete NUMA-optimized branch of the ktransformers project☆27Nov 3, 2025Updated last year
- XTU = X Transformers Unity — hybrid (CPU-expert + GPU) inference acceleration for vLLM: mainline plugin + CPU MoE engine (AVX512-VNNI/BF1…☆16Nov 2, 2026Updated last week
- Personal fork of Kacper’s vLLM-Moet for DeepSeek V4 Flash on a single RTX PRO 6000. Shared for others to use; support is best-effort as t…☆18Aug 19, 2026Updated 2 months ago
- 1Cat-vLLM on Intel Gaudi☆47Oct 12, 2026Updated last month
- A cmpunlocker fork with working p2p on SYS-4028GR-TRT2☆27Aug 8, 2026Updated 3 months ago
- Unlock P2P comms between consumer NVIDIA GPUs☆719Oct 26, 2026Updated 2 weeks ago
- Local LLM Inference Speed Test Tool☆252Oct 21, 2026Updated 3 weeks ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- INT8-native vLLM fork☆28Updated this week
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆206Updated this week
- Reproducible SGLang image for DeepSeek-V4-Flash-0731 on SM120☆38Aug 25, 2026Updated 2 months ago
- GPU peer-to-peer over dynamic BAR1 for GeForce cards: patched NVIDIA open kernel modules.☆19Nov 6, 2026Updated last week
- Create a Mahjong Game with Python☆13Jan 4, 2025Updated last year
- Lynn GitHub 镜像仓 · Qwen3.8-27B Coder390 EfficientThink 本地 GGUF（llama.cpp Q2/Q3，内置 Q4 MTP）· 主仓 https://github.com/MerkyorLynn/Lynn · 下载 htt…☆36Updated this week
- W4A4 full（VLLM_MOE_W4A4=2）+ （VLLM_B12X_SHARED_WRAPPER=1）+ FlashInfer 0.6.16 + threshold 4096 + util 0.78 + DSpark MTP n7 + CUMEM=0☆16Aug 26, 2026Updated 2 months ago
- A simple Gradio WebUI for loading/unloading models and loras in tabbyAPI.☆20Nov 21, 2024Updated last year
- llama.cpp fork with additional SOTA quants and improved performance☆3,387Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A reproducible local LLM benchmarking platform with versioned datasets, deterministic scoring, and auditable reports.☆181Oct 28, 2026Updated 2 weeks ago
- 双 SM80 / CMP 170HX 的 Qwen3.8-Flash-Next 社区 vLLM：NVIDIA NVFP4、GDS、TEP2、MTP6，支持 1～10 并发，词表与 PCIe 通信优化。中英文部署与可构建源码。☆28Oct 15, 2026Updated 3 weeks ago
- Vendor-neutral collective communication for GPU groups without a common native collective library, plus a PCIe/BAR1 direct path for consu…☆13Aug 30, 2026Updated 2 months ago
- Docker images for LLM inference (SGLang + vLLM) on NVIDIA Blackwell GPUs (SM120, CUDA 13.2)☆100Oct 20, 2026Updated 3 weeks ago
- Qwen3.8 Flash Next AWQ INT4 4x3090 v1: 262K, Vision, FP8 QSA KV and CUDA Graphs on 4x RTX 3090☆16Aug 30, 2026Updated 2 months ago
- A tool to unlobotomize your NVIDIA card!☆841Oct 28, 2026Updated 2 weeks ago
- Multimodal gameplay video understanding system combining vision, audio, and language models to enable long-horizon reasoning and question…☆14Oct 14, 2026Updated 3 weeks ago
- GLM-5.3-Flash (320B MoE) at TP4 across four DGX Sparks, same day as the model drop: 36 tok/s, 1.26M-token FP8 KV pool, 262K context, MTP …☆57Oct 10, 2026Updated last month
- Run MoE models beyond VRAM on AMD, NVIDIA & Intel GPUs. Vulkan-powered, portable Windows app with automatic setup. | 让游戏显卡跑起超显存大模型，支持 AMD…☆357Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Qwen4-Exp (Qwen3.8-Flash-Next) SWA + MTP + PLE-on-iswa — bounded deep-context decode w/ long-range recall, TBQ4 4.25 bpv KV, DSpark + Rot…☆94Sep 24, 2026Updated last month
- Large-Language-Model to Machine Interface project.☆18Dec 5, 2023Updated 2 years ago
- llama.cpp fork for running Qwen3.8-Flash-Next V3 (95.5 GiB) on a 64 GB Apple Silicon Mac by streaming MoE experts from SSD. Setup guide: …☆13Sep 19, 2026Updated last month
- AdaLLM is an NVFP4-first inference runtime for Ada Lovelace (RTX 4090) with FP8 KV cache and custom decode kernels. This repo targets NVF…☆156Feb 15, 2026Updated 8 months ago
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,494Updated this week
- a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-…☆485Updated this week
- The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-reques…☆1,215Oct 1, 2026Updated last month
Are these results useful?