Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
☆1,199Nov 15, 2026Updated this week
Alternatives and similar repositories for tiny-vllm
Users that are interested in tiny-vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen☆5,760Updated this week
- Nano vLLM☆16,466Apr 26, 2026Updated 6 months ago
- A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.☆4,374Oct 17, 2026Updated last month
- A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.☆5,609May 17, 2026Updated 6 months ago
- GPT-2-style LLM built from scratch in C/CUDA with hand-written backprop, BPE tokenizer, FlashAttention, pretraining, and SFT.☆127Jun 18, 2026Updated 5 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A generative pretrained transformer implementation☆94Aug 30, 2026Updated 2 months ago
- ☆13Sep 9, 2026Updated 2 months ago
- Region-level profiling for CUDA kernels with trace, NVBit, CUPTI, NSys, and an interactive Explorer.☆135Apr 17, 2026Updated 7 months ago
- GPU programming related news and material links☆2,413Jun 15, 2026Updated 5 months ago
- Speed-of-Light SW efficiency by using ultra low-latency primitives for comms collectives☆13Apr 8, 2026Updated 7 months ago
- Learn CUDA with PyTorch☆534Oct 3, 2026Updated last month
- Core-Aware Selective KV Compression for Reasoning Traces☆11May 21, 2026Updated 5 months ago
- Generate hands-on, multi-part technical tutorials on demand, with LLM skills tuned to make content approachable. Then you work through th…☆1,718Nov 7, 2026Updated last week
- the hacker’s whirlwind tour to deep learning alchemy and deep learning systems☆98Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Autoresearch for GPU kernels. Give it any PyTorch model, go to sleep, wake up to optimized Triton kernels.☆1,630Updated this week
- Two case studies: megakernels roughly on par with CUDA Graphs + PDL. Code and baselines for further research.☆12Nov 6, 2026Updated last week
- TokenSpeed is a speed-of-light LLM inference engine.☆2,934Updated this week
- ☆3,475May 5, 2026Updated 6 months ago
- A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm,…☆44May 17, 2026Updated 6 months ago
- A pure-Python implementation of the Nvidia CuTe layout algebra intended to be approachable and easy to learn.☆259Jun 29, 2026Updated 4 months ago
- Language model tokenization at GB/s☆4,157Nov 2, 2026Updated 2 weeks ago
- DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm☆24,384Nov 8, 2026Updated last week
- Material for gpu-mode lectures☆7,098Sep 9, 2026Updated 2 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference…☆691Apr 16, 2026Updated 7 months ago
- The best Claude Code that $200 can buy☆271Apr 6, 2026Updated 7 months ago
- Optimized GPU compiler for LLM inference. Choose from a list of optimized recipes or optimize your own model via kernel fusion, autotunin…☆125Updated this week
- From-scratch multi-GPU LLM inference engine: paged KV cache, continuous batching, prefix caching, tensor parallelism, OpenAI-compatible s…☆28Sep 25, 2026Updated last month
- ☆11Jan 22, 2026Updated 9 months ago
- Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O☆610Sep 13, 2025Updated last year
- Tile primitives for speedy kernels☆3,803Sep 12, 2026Updated 2 months ago
- FlashInfer: Kernel Library for LLM Serving☆6,958Updated this week
- nCPU: model-native and tensor-optimized CPU research runtimes with organized workloads, tools, and docs☆668Jul 30, 2026Updated 3 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Code, labs, and resources for O'Reilly AI Systems Performance Engineering: GPU optimization, distributed training, inference scaling, and…☆2,228Updated this week
- https://github.com/gpu-mode/reference-kernels☆27Jul 4, 2026Updated 4 months ago
- LMCache: Supercharge Your LLM with the Fastest KV Cache Layer☆12,388Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆97,169Updated this week
- Mirage Persistent Kernel: Compiling LLMs into a MegaKernel☆2,621Updated this week
- Intercept LLM API traffic and visualize token usage in a real-time terminal dashboard. Track costs, debug prompts, and monitor cont…☆812Jun 21, 2026Updated 4 months ago
- Flux 2 image generation model pure C inference☆2,085Oct 23, 2026Updated 3 weeks ago
Are these results useful?