Nano vLLM
☆27Aug 11, 2025Updated last year
Alternatives and similar repositories for nano-vllm
Users that are interested in nano-vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A from-scratch Prefill/Decode disaggregation inference engine for LLMs☆163May 10, 2026Updated 5 months ago
- ☆17Jan 8, 2025Updated last year
- 东南大学的科技读写课的期末作业专利申请书的LaTeX模板☆13Jun 17, 2022Updated 4 years ago
- ☆13Jun 25, 2022Updated 4 years ago
- ☆174Mar 5, 2026Updated 7 months ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- ☆58Updated this week
- 基于RK3588的边缘预警项目（C++）。 采用Yolov7算法，通过读取输入视频源（RTSP、USB摄像头等），经过自创轻量级目标跟踪算法去重检测，将告警信息实时通过HTTP接口推送，同时将视频源进行转发并利用RTMP协议推送。☆17Apr 28, 2024Updated 2 years ago
- Lightweight LLM inference engine inspired by nano-vllm, with radix-tree based prefix cache, tp & pp, cuda graph, openai api, async schedu…☆17Mar 29, 2026Updated 7 months ago
- Advanced Programming - HW5☆23May 10, 2022Updated 4 years ago
- MUSA AI Tensor Engine☆24Oct 14, 2026Updated 2 weeks ago
- Optimize SGEMM step by step☆10Sep 4, 2024Updated 2 years ago
- ☆12Feb 2, 2014Updated 12 years ago
- PyTorch code for our paper "AdaSVD: Adaptive Singular Value Decomposition for Large Language Models"☆16Mar 9, 2025Updated last year
- This repository organizes materials, recordings, and schedules related to AI-infra learning meetings.☆614Mar 1, 2026Updated 7 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Triton based sparse quantization attention kernel collection☆43Aug 29, 2025Updated last year
- Framework for Multimodal Deformable Image Registration. Coordinated equivariant representation learning (CoMIR) combined with robust defo…☆12Mar 7, 2023Updated 3 years ago
- ☆11Mar 21, 2023Updated 3 years ago
- INSPIRE: Intensity and Spatial Information-Based Deformable Image Registration☆12Jun 30, 2021Updated 5 years ago
- nano-vllm是开源的一个gpu推理项目，基于开源版本弄的一个ascend npu版本推理小demo，旨在帮助初学者了解推理的整体流程，区别于vllm，nano-vllm体量更小，麻雀虽小五脏俱全，更有助于初学者学习。☆97May 4, 2026Updated 5 months ago
- 基于Rust的大模型推理框架☆14Oct 21, 2026Updated last week
- A lightweight CuTe-based CUDA kernel for DuoAttention, optimized for large language model inference.☆12Dec 25, 2025Updated 10 months ago
- LW-CTrans☆18Jun 4, 2024Updated 2 years ago
- Logic Reinforcement Learning☆21Oct 20, 2025Updated last year
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- GLM-5.3-Flash adaptation + offload perf patches for FreeToken v0.1.2 (17.8→36.7 tok/s on one RTX PRO 6000)☆20Oct 2, 2026Updated 3 weeks ago
- AI 会议助手☆43Sep 29, 2026Updated last month
- PyTorch Implementation of the CVPR'25 Paper "Fitted Neural Lossless Image Compression"☆20Aug 5, 2026Updated 2 months ago
- 基于RK3588S的4K边缘视觉网关：支持V4L2/RTMP/RTSP视频源、RKNN YOLOv8s推理、RGA+MPP硬件加速、RTMP推流、MQTT上报及多路NPU调度。☆38Aug 24, 2026Updated 2 months ago
- 这是一个基于C++实现的从零开始的大模型推理框架☆10Nov 18, 2024Updated last year
- 2024华为嵌入式软件大赛-算法组 冠军队伍题解☆13Aug 5, 2024Updated 2 years ago
- ☆13May 18, 2026Updated 5 months ago
- Based on Nano-vLLM, a simple replication of vLLM with self-contained paged attention and flash attention implementation☆1,096Aug 29, 2026Updated 2 months ago
- High-performance RL post-training infrastructure. Designed to achieve bitwise operator-level train-inference consistency across heterogen…☆444Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Implementation of DAC'22 paper: Hierarchical Memory-Constrained Operator Scheduling of Neural Architecture Search Networks.☆11Sep 5, 2022Updated 4 years ago
- 通过onnxruntime实现yolov8在CPU和GPU上面部署☆27Aug 17, 2024Updated 2 years ago
- A visualization tool to display TF-Grappler optimized op graph☆13Aug 31, 2022Updated 4 years ago
- Using acl and c/c++ writing coroutine,http/https,server,json,redis,mysql,network,nio,etc,.☆20Oct 8, 2025Updated last year
- 注释的nano_vllm仓库，并且完成了MiniCPM4的适配以及注册新模型的功能☆211Aug 11, 2025Updated last year
- 跟我一起写 Makefile (Markdown重制版 )☆15May 29, 2024Updated 2 years ago
- 新一代智能工业/科研平台☆22Aug 17, 2026Updated 2 months ago