vllm-project / vllm-ascendLinks
Community maintained hardware plugin for vLLM on Ascend
☆1,597Updated this week
Alternatives and similar repositories for vllm-ascend
Users that are interested in vllm-ascend are comparing it to the libraries listed below
Sorting:
- A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.☆2,329Updated last week
- High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.☆1,387Updated last week
- RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.☆1,031Updated last week
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.☆4,632Updated this week
- The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.☆1,517Updated last month
- FlagScale is a large model toolkit based on open-sourced projects.☆468Updated last week
- Ascend PyTorch adapter (torch_npu). Mirror of https://gitee.com/ascend/pytorch☆481Updated last week
- 一种任务级GPU算力分时调度的高性能深度学习训练平台☆731Updated 2 years ago
- vLLM Kunlun (vllm-kunlun) is a community-maintained hardware plugin designed to seamlessly run vLLM on the Kunlun XPU.☆239Updated this week
- Lightning-Fast RL for LLM Reasoning and Agents. Made Simple & Flexible.☆3,446Updated this week
- ☆522Updated last week
- LLM 并发性能测试工具，支持自动化压力测试和性能报告生成。☆211Updated last month
- 通义千问VLLM推理部署DEMO☆637Updated last year
- FlashInfer: Kernel Library for LLM Serving☆4,768Updated this week
- LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.☆859Updated last month
- Disaggregated serving system for Large Language Models (LLMs).☆772Updated 9 months ago
- DLRover: An Automatic Distributed Deep Learning System☆1,626Updated last week
- ☆182Updated this week
- [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.☆669Updated 2 months ago
- LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalabili…☆3,860Updated this week
- An Efficient and User-Friendly Scaling Library for Reinforcement Learning with Large Language Models☆2,711Updated this week
- Train a 1B LLM with 1T tokens from scratch by personal☆786Updated 9 months ago
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.☆659Updated this week
- A self-learning tutorail for CUDA High Performance Programing.☆854Updated 2 weeks ago
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆2,615Updated last week
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆274Updated 5 months ago
- My learning notes for ML SYS.☆5,157Updated last week
- slime is an LLM post-training framework for RL Scaling.☆3,571Updated this week
- vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization☆2,122Updated last week
- VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo☆1,576Updated this week