Nano vLLM v1 engine
☆15Aug 6, 2025Updated last year
Alternatives and similar repositories for nano-vllm-v1
Users that are interested in nano-vllm-v1 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Radix Tree KV Cache with Turn-Aware Growth☆12Jul 21, 2025Updated last year
- Lightweight LLM inference engine inspired by nano-vllm, with radix-tree based prefix cache, tp & pp, cuda graph, openai api, async schedu…☆17Mar 29, 2026Updated 7 months ago
- Nano vLLM Triton☆14Aug 27, 2025Updated last year
- A replication of "Toy Models of Superposition," a groundbreaking machine learning research paper published by authors affiliated with Ant…☆11Dec 30, 2023Updated 2 years ago
- Task scheduling based on thread pool. Getting parallel tasks for a given dependency graph (or task graph), which is a DAG, via topologica…☆11Mar 21, 2017Updated 9 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Nano vLLM with vLLM v1's request scheduling strategy and chunked prefill☆105Jan 26, 2026Updated 9 months ago
- ☆16Aug 26, 2026Updated 2 months ago
- Code and data repository for two papers (ACL & EMNLP 2024) on the topic of collapse in model editing.☆10Dec 20, 2024Updated last year
- ☆17May 10, 2024Updated 2 years ago
- Implementation for ACL 2024 paper "Meta-Task Prompting Elicits Embeddings from Large Language Models"☆12Jul 25, 2024Updated 2 years ago
- PilotFish harvests the free GPU cycles of cloud gaming with deep learning training☆14Jul 2, 2022Updated 4 years ago
- Provides a minimal implementation to extract FLAN datasets for further processing☆11Feb 1, 2023Updated 3 years ago
- 龙芯杯2021个人赛决赛最终代码☆11Sep 1, 2021Updated 5 years ago
- ROS 2 Inference sample for using Rockchip NPU.☆17Feb 20, 2023Updated 3 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Evaluation scripts for the Open TTS Leaderboard: https://huggingface.co/spaces/hf-audio/open_tts_leaderboard☆15Updated this week
- 重庆大学计算机学院2018级计算机体系结构cache设计☆11Jan 4, 2021Updated 5 years ago
- This code implements a basic, Twitter-aware tokenizer.☆12Feb 8, 2024Updated 2 years ago
- High-performance CUDA kernels for real-time financial low latency inference, optimized for both consumer and datacenter GPUs.☆20Jul 25, 2025Updated last year
- ML framework featuring compile time checks and accelerated by a JIT compiler.☆10Mar 19, 2026Updated 7 months ago
- 🧩 Claude Code Neovim IDE Extension☆11Apr 30, 2026Updated 6 months ago
- Blazing-fast LLM inference in pure Rust. No PyTorch and Python runtime.☆354Oct 31, 2026Updated 2 weeks ago
- NVIDIA Deepstream SDK & Gstreamer excercise☆20Apr 6, 2021Updated 5 years ago
- Dynamic Spear Model☆12Jul 24, 2019Updated 7 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- This repository contains the code for the paper "LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Sig…☆27Apr 15, 2026Updated 7 months ago
- FlashFunAsr: A lightweight vLLM implementation built from scratch for FunASR nano☆12Jan 5, 2026Updated 10 months ago
- A survey of modern quantization formats (e.g., MXFP8, NVFP4) and inference optimization tools (e.g., TorchAO, GemLite), illustrated throu…☆12Nov 11, 2025Updated last year
- Code for paper "Spider: Any-to-Many Multimodal LLM"☆16Apr 26, 2025Updated last year
- CUDA Graph runtime tensor debugging utilities for PyTorch☆18Jul 12, 2026Updated 4 months ago
- Enhanced version of original AutoGPTQ (https://github.com/PanQiWei/AutoGPTQ).☆10Nov 2, 2023Updated 3 years ago
- LightRFT (Light Reinforcement Fine-Tuning) is an advanced reinforcement learning fine-tuning framework designed for Large Language Models…☆19Jan 12, 2026Updated 10 months ago
- 更纯粹、更高压缩率的Tokenizer in Rust☆14Sep 7, 2026Updated 2 months ago
- 关于无锁队列的知识☆11Feb 13, 2017Updated 9 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Code and data repository for "The Mirage of Model Editing: Revisiting Evaluation in the Wild"☆18Aug 27, 2025Updated last year
- [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models☆23Mar 15, 2024Updated 2 years ago
- Code for the paper "A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis"☆20Jun 12, 2025Updated last year
- Graph model execution API for Candle☆19Jul 27, 2025Updated last year
- Examples from the Learning Burn book.☆17Oct 14, 2026Updated last month
- ☆11Jul 21, 2026Updated 3 months ago
- ViTALiTy (HPCA'23) Code Repository☆25Mar 13, 2023Updated 3 years ago
Are these results useful?