vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill
☆34Nov 7, 2025Updated last year
Alternatives and similar repositories for exvllm
Users that are interested in exvllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 使用nvidia oam转接卡将intel gaudi2转接到pcie并在ubuntu下正常驱动和使用的指南☆31Nov 4, 2026Updated last week
- The complete NUMA-optimized branch of the ktransformers project☆27Nov 3, 2025Updated last year
- forked from vllm-project/flash-attention☆74May 9, 2026Updated 6 months ago
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆168Updated this week
- INT8-native vLLM fork☆28Updated this week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Local LLM Inference Speed Test Tool☆252Oct 21, 2026Updated 3 weeks ago
- KTransformers 一键部署脚本☆61Apr 18, 2025Updated last year
- DETR tensor去除推理过程无用辅助头+fp16部署再次加速+解决转tensorrt 输出全为0问题的新方法。☆12Jan 9, 2024Updated 2 years ago
- DeepSeek-V4-Flash on Ampere SM 8.6 via vLLM (pyref kernel replacements)☆36Jul 2, 2026Updated 4 months ago
- ☆974Sep 14, 2025Updated last year
- 设置Windows下英伟达计算卡的工作模式。Change Nvidia Tesla GPU work mode to WDDM on Windows.☆13Oct 15, 2023Updated 3 years ago
- Skywork Reward Model Series☆10Sep 6, 2024Updated 2 years ago
- run DeepSeek-R1 GGUFs on KTransformers☆256Mar 3, 2025Updated last year
- fastllm是后端无依赖的高性能大模型推理库，帮助您在大部分设备上高效推理大模型！☆5,130Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆28Oct 20, 2026Updated 3 weeks ago
- This project is specifically developed for V100, based on lmdeploy 0.12.1, and supports mainstream open-source models from Q4 2025 to Q1 …☆22Mar 18, 2026Updated 7 months ago
- yolo-pose for training escalator data☆16Jul 1, 2024Updated 2 years ago
- 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.☆10Nov 19, 2024Updated last year
- ☆19Aug 23, 2022Updated 4 years ago
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区，聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆26Jul 2, 2026Updated 4 months ago
- Generates kid-friendly weather forecasts suitable for display on a large monitor. Uses OpenWeatherMap and a local or hosted LLM.☆17Feb 28, 2026Updated 8 months ago
- Frequency-based KV cache pruning for llama.cpp — 25% cache reduction, improved PPL at long context. GPU compaction kernel for HIP/ROCm.☆16Apr 18, 2026Updated 6 months ago
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆206Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆67Feb 15, 2026Updated 8 months ago
- 大语言模型工具集☆28Aug 1, 2025Updated last year
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆61Sep 16, 2026Updated last month
- This project is intended to build and deploy an SNPE model on Qualcomm Devices, which are having unsupported layers which are not part of…☆10Oct 4, 2021Updated 5 years ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆12Oct 28, 2026Updated 2 weeks ago
- MobileSAM のエンコーダー/デコーダーをONNXに変換し、推論するサンプル☆12Apr 11, 2024Updated 2 years ago
- Linux P2P unlock and performance fixes for NVIDIA CMP 170HX☆39Sep 22, 2026Updated last month
- A cmpunlocker fork with working p2p on SYS-4028GR-TRT2☆27Aug 8, 2026Updated 3 months ago
- a repo to study TensorRT Plugin☆11Apr 30, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- TensorRT-FastSAM(https://github.com/CASIA-IVA-Lab/FastSAM)☆23Feb 29, 2024Updated 2 years ago
- Converts CLIP models to ONNX☆11Jan 17, 2023Updated 3 years ago
- 1Cat Studio: unified vLLM workbench, Codex Agent and creative canvas☆15Updated this week
- Inference engine for large language models on consumer GPUs, with deep optimization for older hardware☆55Oct 20, 2026Updated 3 weeks ago
- ragflow中的ocr部分，非官方项目☆54Aug 26, 2024Updated 2 years ago
- ☆23Dec 15, 2016Updated 9 years ago
- Revision of official yolov7-pose to support custom dataset for keypoint detection☆11Nov 12, 2023Updated 3 years ago
Are these results useful?