Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
☆1,166Oct 22, 2026Updated this week
Alternatives and similar repositories for tiny-vllm
Users that are interested in tiny-vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.☆4,282Oct 17, 2026Updated last week
- learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen☆4,788Updated this week
- GPT-2-style LLM built from scratch in C/CUDA with hand-written backprop, BPE tokenizer, FlashAttention, pretraining, and SFT.☆126Jun 18, 2026Updated 4 months ago
- Nano vLLM☆16,121Apr 26, 2026Updated 6 months ago
- A generative pretrained transformer implementation☆94Aug 30, 2026Updated last month
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Enlightener, the cutting-edge Retrieval-Augmented Generation (RAG) system that revolutionizes query responses. By combining the power of …☆13Jul 28, 2025Updated last year
- ☆13Sep 9, 2026Updated last month
- A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.☆5,431May 17, 2026Updated 5 months ago
- A curated list of best cuda programming books☆994May 19, 2026Updated 5 months ago
- From-scratch multi-GPU LLM inference engine: paged KV cache, continuous batching, prefix caching, tensor parallelism, OpenAI-compatible s…☆28Sep 25, 2026Updated last month
- Region-level profiling for CUDA kernels with trace, NVBit, CUPTI, NSys, and an interactive Explorer.☆133Apr 17, 2026Updated 6 months ago
- Generate hands-on, multi-part technical tutorials on demand, with LLM skills tuned to make content approachable. Then you work through th…☆1,705Oct 18, 2026Updated last week
- a whirlwind tour to deep learning alchemy and deep learning systems☆95Updated this week
- TokenSpeed is a speed-of-light LLM inference engine.☆2,782Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A first-principles climb from the absolute bottom of hardware to the top of GPU architecture.☆103Oct 6, 2026Updated 2 weeks ago
- GPU programming related news and material links☆2,391Jun 15, 2026Updated 4 months ago
- Autoresearch for GPU kernels. Give it any PyTorch model, go to sleep, wake up to optimized Triton kernels.☆1,605Updated this week
- Language model tokenization at GB/s☆4,136Sep 2, 2026Updated last month
- A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm,…☆44May 17, 2026Updated 5 months ago
- ☆3,432May 5, 2026Updated 5 months ago
- A pure-Python implementation of the Nvidia CuTe layout algebra intended to be approachable and easy to learn.☆253Jun 29, 2026Updated 3 months ago
- The best Claude Code that $200 can buy☆269Apr 6, 2026Updated 6 months ago
- Flash-Flash KDA: H100-optimized Flash Kimi Delta Attention kernels☆24Jul 24, 2026Updated 3 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference…☆689Apr 16, 2026Updated 6 months ago
- Learn CUDA with PyTorch☆519Oct 3, 2026Updated 3 weeks ago
- Algebra engine.☆60Aug 14, 2026Updated 2 months ago
- DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm☆24,071Updated this week
- Material for gpu-mode lectures☆6,911Sep 9, 2026Updated last month
- Optimized GPU compiler for LLM inference. Choose from a list of optimized recipes or optimize your own model via kernel fusion, autotunin…☆117Updated this week
- nCPU: model-native and tensor-optimized CPU research runtimes with organized workloads, tools, and docs☆662Jul 30, 2026Updated 2 months ago
- Batched square compact-Householder QR factorization.☆15Jul 2, 2026Updated 3 months ago
- Tile primitives for speedy kernels☆3,783Sep 12, 2026Updated last month
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- https://github.com/gpu-mode/reference-kernels☆27Jul 4, 2026Updated 3 months ago
- A re-implementation of Karpathy's micrograd in C☆66Jun 21, 2026Updated 4 months ago
- Code, labs, and resources for O'Reilly AI Systems Performance Engineering: GPU optimization, distributed training, inference scaling, and…☆2,109Updated this week
- Intercept LLM API traffic and visualize token usage in a real-time terminal dashboard. Track costs, debug prompts, and monitor cont…☆810Jun 21, 2026Updated 4 months ago
- LMCache: Supercharge Your LLM with the Fastest KV Cache Layer☆12,164Updated this week
- PyTorch implementations of modern open-source LLM architectures (Llama, Qwen, DeepSeek, Gemma, GPT-OSS, Kimi, and more) — written from sc…☆426Sep 30, 2026Updated 3 weeks ago
- Shadow any website for offline viewing, with the JavaScript stripped out☆3,494Aug 10, 2026Updated 2 months ago