nano-vllm是开源的一个gpu推理项目，基于开源版本弄的一个ascend npu版本推理小demo，旨在帮助初学者了解推理的整体流程，区别于vllm，nano-vllm体量更小，麻雀虽小五脏俱全，更有助于初学者学习。
☆95May 4, 2026Updated 6 months ago
Alternatives and similar repositories for nano-vllm-ascend
Users that are interested in nano-vllm-ascend are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Radix Tree KV Cache with Turn-Aware Growth☆12Jul 21, 2025Updated last year
- Nano vLLM Triton☆14Aug 27, 2025Updated last year
- 注释的nano_vllm仓库，并且完成了MiniCPM4的适配以及注册新模型的功能☆212Aug 11, 2025Updated last year
- nano-vllm 面试导向学习指南 | 20节课程 + 72道面试八股文 + STAR面试法 + 哆啦A梦风格漫画☆311Apr 2, 2026Updated 7 months ago
- PyPTO kernels and end-to-end LLM models for Ascend NPUs.☆27Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Community maintained hardware plugin for vLLM on Huawei Ascend☆3,077Updated this week
- Nano vLLM☆27Aug 11, 2025Updated last year
- 面向开发者与初学者的 nano-vLLM 交互式源码教程 - 通过 13 个 HTML 互动实验 + 13 章中文教程理解大模型推理引擎☆41Aug 23, 2026Updated 2 months ago
- ☆12Nov 12, 2024Updated 2 years ago
- Omni_Infer is a suite of inference accelerators designed for the Ascend NPU platform, offering native support and an expanding feature se…☆133Sep 20, 2026Updated last month
- LMCache-Ascend is a plugin for running LMCache on the Ascend NPU.☆93Updated this week
- ☆30Updated this week
- Pure Rust SSTable implementation☆14Aug 9, 2024Updated 2 years ago
- Visualize the Expert Parallelism Load Balancer☆19Mar 15, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- CUDA MoE kernels.☆12Sep 21, 2024Updated 2 years ago
- ☆19Apr 15, 2025Updated last year
- ☆21Jun 9, 2025Updated last year
- 中山大学学生适用的简历模板☆13Jun 6, 2025Updated last year
- A high-performance and light-weight router for vLLM large scale deployment☆512Updated this week
- Accurate and Robust Object SLAM with 3D Quadric Landmark Reconstruction in Outdoors☆21Mar 2, 2022Updated 4 years ago
- Persist and reuse KV Cache to speedup your LLM.☆354Updated this week
- Projects Pages for the NJU-3DV's Reserach Work☆10May 8, 2026Updated 6 months ago
- UCB《Operating Systems and Systems Programming》课程：https://cs162.eecs.berkeley.edu☆31May 4, 2016Updated 10 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆29Jan 13, 2024Updated 2 years ago
- Repo for "STAR: Spectral Truncation and Rescale for Model Merging" [NAACL 2025]☆11Oct 22, 2026Updated 3 weeks ago
- Visual odometry (based on image intensity) implementation on CUDA☆24Oct 16, 2018Updated 8 years ago
- ☆40Jul 27, 2026Updated 3 months ago
- VUA stands for 'VAST Undivided Attention'. It's a global KVCache storage solution optimizing LLM time to first token (TTFT) and GPU utili…☆38Mar 12, 2026Updated 8 months ago
- Description: Experimental nano-vLLM fork for Qwen3.6 support, CUDA Graph, paged KV cache, and inference-engine learning.☆20Oct 15, 2026Updated 3 weeks ago
- ☆12Apr 29, 2022Updated 4 years ago
- Curated collection of papers in MoE model inference☆431Mar 12, 2026Updated 8 months ago
- ☆56Oct 16, 2026Updated 3 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆12Feb 2, 2014Updated 12 years ago
- ☆12Dec 17, 2022Updated 3 years ago
- TokenSim is a tool for simulating the behavior of large language models (LLMs) in a distributed environment.☆48Nov 4, 2026Updated last week
- A single-file educational implementation for understanding vLLM's core concepts and running LLM inference.☆47Apr 7, 2026Updated 7 months ago
- Nano vLLM v1 engine☆15Aug 6, 2025Updated last year
- ☆11Nov 7, 2023Updated 3 years ago
- Flash Attention in raw Cuda C beating PyTorch☆39May 14, 2024Updated 2 years ago
Are these results useful?