Building an LLM inference engine from scratch
☆29Jul 18, 2026Updated 4 months ago
Alternatives and similar repositories for nano-vllm
Users that are interested in nano-vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- CuratorKIT : Data curation and synthetic data generation for LLM post-training.☆37Updated this week
- An external memory for LLMs: a bi-temporal knowledge graph with hybrid (keyword + vector) retrieval and a self-improving reflection loop.…☆23Updated this week
- DL Backtrace is a new explainablity technique for deep learning models that works for any modality and model type.☆28Updated this week
- A complete PyTorch implementation of Google's Gemma3 270M language model, featuring sliding window attention, RoPE positional encoding, a…☆49Sep 7, 2025Updated last year
- Aligntune : A Modular Toolkit for Post Training Alignment of LLMs☆49Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆17Aug 22, 2026Updated 2 months ago
- A vector field rendering library☆17Jul 31, 2019Updated 7 years ago
- 手写 Triton kernel 实现的 Qwen3.5-0.8B 纯文本推理引擎(batch=1, BF16, greedy)☆12Sep 10, 2026Updated 2 months ago
- Benchmark and controlled study of LLM-based novelty judges for research ideas. Code and data for "Old Ideas, Novel Problems" (arXiv 2610.…☆21Oct 26, 2026Updated 3 weeks ago
- ☆25Jun 30, 2021Updated 5 years ago
- High-performance GEMM implementation optimized for NVIDIA H100 GPUs, leveraging Hopper architecture's TMA, WGMMA, and Thread Block Cluste…☆11Aug 25, 2026Updated 2 months ago
- Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMs☆24Updated this week
- ☆12Oct 18, 2022Updated 4 years ago
- ☆13Jan 7, 2026Updated 10 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- ☆14Dec 22, 2024Updated last year
- ☆16Dec 2, 2025Updated 11 months ago
- ☆19Nov 5, 2025Updated last year
- Generating Summaries with Controllable Readability Levels (EMNLP 2023)☆15Jul 2, 2026Updated 4 months ago
- Implementation of the Benchmark Approaches for Medical Instructional Video Classification (MedVidCL) and Medical Video Question Answering…☆31Jan 31, 2023Updated 3 years ago
- stable diffusion using pytorch only☆11Apr 29, 2026Updated 6 months ago
- ☆16Oct 21, 2025Updated last year
- ☆25Jan 14, 2025Updated last year
- Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"☆32Mar 28, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆11Oct 15, 2024Updated 2 years ago
- Radix Tree KV Cache with Turn-Aware Growth☆12Jul 21, 2025Updated last year
- This project applies multiple deep learning models to the problem of restoring diacritical marks to sentences in Vietnamese.☆25Nov 13, 2018Updated 8 years ago
- GStar Bootcamp - Assignment 1☆22Sep 8, 2025Updated last year
- Deep Dream Machine iOS☆11Jul 12, 2015Updated 11 years ago
- Max for Live experiments☆12Mar 23, 2015Updated 11 years ago
- FHBI: Improving Generalization with Flat Hilbert Bayesian Inference (ICML2025)☆19Jan 7, 2026Updated 10 months ago
- Preserving Generalization of Language Models in Few-shot Continual Relation Extraction (EMNLP2024)☆17Nov 21, 2024Updated last year
- ☆10Nov 9, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Discover, evaluate, and intervene on circuits in transformer models.☆32Updated this week
- EleutherAI ML Performance reading group repository (slides, meeting recordings, annotated papers)☆38Mar 20, 2026Updated 8 months ago
- Minimal uart for small iCE40 Lattice FPGAs. 8N1. 32 or 34 logicCells☆10Jul 20, 2017Updated 9 years ago
- From-scratch multi-GPU LLM inference engine: paged KV cache, continuous batching, prefix caching, tensor parallelism, OpenAI-compatible s…☆28Sep 25, 2026Updated last month
- Variational Recurrent Auto Encoder☆16Jul 10, 2016Updated 10 years ago
- GSoC'16 RedHen Labs☆11Aug 22, 2016Updated 10 years ago
- Distil low resource language knowledge from large to small language models☆12Jan 28, 2026Updated 9 months ago
Are these results useful?