vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill
☆34Nov 7, 2025Updated 8 months ago
Alternatives and similar repositories for exvllm
Users that are interested in exvllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- forked from vllm-project/flash-attention☆58May 9, 2026Updated 2 months ago
- Local LLM Inference Speed Test Tool☆123Jul 2, 2026Updated 3 weeks ago
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区，聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆21Jul 2, 2026Updated 3 weeks ago
- ☆14Feb 3, 2022Updated 4 years ago
- This project is specifically developed for V100, based on lmdeploy 0.12.1, and supports mainstream open-source models from Q4 2025 to Q1 …☆21Mar 18, 2026Updated 4 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- yolo-pose for training escalator data☆16Jul 1, 2024Updated 2 years ago
- run DeepSeek-R1 GGUFs on KTransformers☆258Mar 3, 2025Updated last year
- Use OpenCode in WeChat☆22Mar 22, 2026Updated 4 months ago
- vLLM fork for Tesla V100 (SM70) with AWQ 4-bit support, CUDA 12.8 build flow, and validated Qwen3.5 27B/35B deployment on multi-GPU V…☆557Jul 21, 2026Updated last week
- TensorRT-FastSAM(https://github.com/CASIA-IVA-Lab/FastSAM)☆23Feb 29, 2024Updated 2 years ago
- 大语言模型工具集☆28Aug 1, 2025Updated 11 months ago
- Vstream - Video Analytics pipeline with Hardware based accelerations (dev - stage)☆10Feb 2, 2024Updated 2 years ago
- This project is intended to build and deploy an SNPE model on Qualcomm Devices, which are having unsupported layers which are not part of…☆10Oct 4, 2021Updated 4 years ago
- MobileSAM のエンコーダー/デコーダーをONNXに変換し、推論するサンプル☆12Apr 11, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 主要写er-nerf从零到一所有部署过程☆44Aug 28, 2024Updated last year
- A "standard library" of Triton kernels.☆26Oct 2, 2025Updated 9 months ago
- Excel RTD server sourcing data from Redis☆11Dec 11, 2024Updated last year
- A C++ implementation for UCMCTrack (SOTA in MOT17)☆27May 30, 2025Updated last year
- Revision of official yolov7-pose to support custom dataset for keypoint detection☆11Nov 12, 2023Updated 2 years ago
- ☆10Jul 18, 2024Updated 2 years ago
- Bert TensorRT模型加速部署☆10Apr 1, 2022Updated 4 years ago
- fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tp…☆4,873Updated this week
- Multiple Lidar preprocessor for BEVfusion☆11Aug 25, 2023Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Precision Knowledge Editing (PKE): A novel method to reduce toxicity in LLMs while preserving performance, with robust evaluations and ha…☆12Nov 26, 2024Updated last year
- ffmpeg+cuvid+tensorrt+multicamera☆12Dec 31, 2024Updated last year
- An implementation of MSSRM method☆10Mar 23, 2023Updated 3 years ago
- 天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案☆50Aug 16, 2023Updated 2 years ago
- Python scripts performing Open Vocabulary Object Detection using the YOLO-World model in ONNX. And Export the ONNX model for AXera's NPU☆12Aug 11, 2025Updated 11 months ago
- This repository provides tutorial, which discusses running sample publisher and subscriber using multiple transports of point_cloud_trans…☆11Mar 17, 2026Updated 4 months ago
- a tiny, portable, stackless coroutine in C++11☆11May 17, 2023Updated 3 years ago
- lightNet (Object Detection and Semantic Segmentation) for ONNX and TensorRT☆16Jul 4, 2023Updated 3 years ago
- 1CatV2 with TileLANG written FA-v100 and many goodies☆18Jul 15, 2026Updated 2 weeks ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Stable Diffusion in TensorRT 8.5+☆15Mar 19, 2023Updated 3 years ago
- Try to export the ONNX QDQ model that conforms to the AXERA NPU quantization specification. Currently, only w8a8 is supported.☆11Sep 10, 2024Updated last year
- A dedicated Colab notebooks to experiment (Nanonets OCR, Monkey OCR, OCRFlux 3B, Typhoo OCR 3B & more..) On T4 GPU - free tier☆26Feb 12, 2026Updated 5 months ago
- FastSAM 部署rknn C++ 代码☆13May 30, 2024Updated 2 years ago
- 完整开源！Java快速开发平台！基于Spring、SpringMVC、Mybatis架构，MStore提供更多好用的插件与模板（文章、商城、微信、论坛、会员、评论、支付、积分、工作流、任务调度等，同时提供上百套免费模板任意选择），价值源自分享！铭飞系统不仅一套简单好用的开源…☆12Jun 4, 2018Updated 8 years ago
- 重构nerf代码，更加容易读懂☆13Mar 26, 2023Updated 3 years ago
- ☆15Jul 23, 2025Updated last year