A minimal implementation of vllm.
☆73Jul 27, 2024Updated 2 years ago
Alternatives and similar repositories for vllmini
Users that are interested in vllmini are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- a minimal cache manager for PagedAttention, on top of llama3.☆151Aug 26, 2024Updated 2 years ago
- ☆92Jan 23, 2025Updated last year
- Cross-GPU KV Cache Marketplace☆27Nov 12, 2025Updated 11 months ago
- minimal C implementation of speculative decoding based on llama2.c☆30Jul 15, 2024Updated 2 years ago
- Proteus: A High-Throughput Inference-Serving System with Accuracy Scaling☆14Mar 7, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆38Jul 28, 2026Updated 3 months ago
- ☆20Jan 27, 2025Updated last year
- 100 days of CUDA Challenge☆49Aug 2, 2025Updated last year
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆364Jun 10, 2025Updated last year
- ☆27Aug 25, 2023Updated 3 years ago
- An ML Systems Onboarding list☆1,150Feb 19, 2026Updated 8 months ago
- Easy, Fast, and Scalable Multimodal AI☆135Jun 2, 2026Updated 5 months ago
- 🎉My Collections of CUDA Kernels~☆11Jun 25, 2024Updated 2 years ago
- GPU kernel benchmarking☆52Oct 10, 2026Updated 3 weeks ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Pytorch Code for---Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factuality.☆16Oct 13, 2025Updated last year
- Implementation of efficient test-time alignment methods.☆14Jan 13, 2026Updated 9 months ago
- LMCache-Ascend is a plugin for running LMCache on the Ascend NPU.☆93Updated this week
- GPTQ inference Triton kernel☆321May 18, 2023Updated 3 years ago
- ☆11Nov 16, 2024Updated last year
- PipeRAG: Fast Retrieval-Augmented Generation via Algorithm-System Co-design (KDD 2025)☆32Jun 14, 2024Updated 2 years ago
- High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.☆130Jul 13, 2024Updated 2 years ago
- Early-stage Rust drop-in alternative frontend for vLLM☆73May 22, 2026Updated 5 months ago
- Awesome-LLM-Productization: a curated list of tools/tricks/news/regulations about AI and Large Language Model (LLM) productization☆24Jan 23, 2025Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Companion code for The Physics of LLM Inference book☆30Apr 21, 2026Updated 6 months ago
- Collection of kernels written in Triton language☆201Jan 27, 2026Updated 9 months ago
- Flash Attention in ~100 lines of CUDA (forward pass only)☆12Jun 10, 2024Updated 2 years ago
- A throughput-oriented high-performance serving framework for LLMs☆976Mar 29, 2026Updated 7 months ago
- A lightweight `vLLM-Omni`-style diffusion implementation built around `Wan2.2-TI2V-5B-Diffusers` inspired from nano-vllm☆65May 25, 2026Updated 5 months ago
- This repository provides code for the paper "Memory-Efficient Fine-Tuning of Transformers via Token Selection" (EMNLP 2024).☆14Feb 11, 2025Updated last year
- A curated list of awesome projects and papers for distributed training or inference☆280Oct 8, 2024Updated 2 years ago
- ☆11Mar 9, 2022Updated 4 years ago
- MobiSys#114☆23Aug 17, 2023Updated 3 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- [COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding☆280Aug 31, 2024Updated 2 years ago
- python-based eDSL for efficient Metal Shading Language code generation☆15Oct 5, 2026Updated last month
- 用pytorch训练18层残差神经网络，用pyqt设计界面☆12Jun 23, 2020Updated 6 years ago
- mKernel: fast multi-node, multi-GPU fused kernels☆308Oct 17, 2026Updated 3 weeks ago
- ☆20Dec 24, 2024Updated last year
- An Attention Superoptimizer☆22Jan 20, 2025Updated last year
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆285Mar 6, 2025Updated last year