注释的nano_vllm仓库，并且完成了MiniCPM4的适配以及注册新模型的功能
☆185Aug 11, 2025Updated 8 months ago
Alternatives and similar repositories for nano_vllm_note
Users that are interested in nano_vllm_note are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- some hpc project for learning☆27Aug 28, 2024Updated last year
- This project aims to provide a high effective KV cache manage framework for llm inference and improve memory utilization and inference sp…☆52Apr 24, 2026Updated 2 weeks ago
- ☆13Jan 7, 2025Updated last year
- Code for paper: "Executing Arithmetic: Fine-Tuning Large Language Models as Turing Machines"☆11Oct 11, 2024Updated last year
- A light llama-like llm inference framework based on the triton kernel.☆184Jan 5, 2026Updated 4 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- EasyNN是一个面向教学而开发的神经网络推理框架，旨在让大家0基础也能自主完成推理框架编写！☆39Aug 26, 2024Updated last year
- ☆23Aug 20, 2025Updated 8 months ago
- Nano vLLM☆13Jun 26, 2025Updated 10 months ago
- An onnx-based quantitation tool.☆71Jan 8, 2024Updated 2 years ago
- ToyLLM: Learning LLM from Scratch☆25Apr 27, 2026Updated last week
- A GPU FP32 computation method with Tensor Cores.☆26Dec 8, 2025Updated 5 months ago
- Implement Flash Attention using Cute.☆106Dec 17, 2024Updated last year
- 跟着Tensorrt_pro学习各种知识☆39Nov 25, 2022Updated 3 years ago
- Optimize softmax in triton in many cases☆24Sep 6, 2024Updated last year
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- 本仓库在OpenVINO推理框架下部署Nanodet检测算法，并重写预处理和后处理部分，具有超高性能！让你在Intel CPU平台上的检测速度起飞！ 并基于NNCF和PPQ工具将模型量化(PTQ)至int8精度，推理速度更快！☆16Jun 14, 2023Updated 2 years ago
- Matrix multiplication on GPUs for matrices stored on a CPU. Similar to cublasXt, but ported to both NVIDIA and AMD GPUs.☆32Apr 2, 2025Updated last year
- ☆32Jul 2, 2025Updated 10 months ago
- A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.☆4,094Mar 13, 2026Updated last month
- 算子库☆17Jul 9, 2025Updated 10 months ago
- patches for huggingface transformers to save memory☆37Jun 2, 2025Updated 11 months ago
- 基于天池新闻推荐赛数据集实现的新闻推荐☆49Dec 17, 2024Updated last year
- a base36 extension for postgres☆18Mar 14, 2025Updated last year
- Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding☆100Dec 2, 2025Updated 5 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A Triton-only attention backend for vLLM☆25Mar 17, 2026Updated last month
- My submission for the GPUMODE/AMD fp8 mm challenge☆29Jun 4, 2025Updated 11 months ago
- AiTer Optimized Model☆71Updated this week
- ☆99May 31, 2025Updated 11 months ago
- study of cutlass☆22Nov 10, 2024Updated last year
- HPC Game Platform☆11Apr 20, 2023Updated 3 years ago
- ☆120May 16, 2025Updated 11 months ago
- A Toolkit for Fine-Tuning Large Language Models with LoRA and DeepSpeed☆11Apr 14, 2023Updated 3 years ago
- flash attention 优化日志☆29Jun 4, 2025Updated 11 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆49Apr 15, 2024Updated 2 years ago
- jump to a place when progam runs to the max instruction number☆15Dec 14, 2023Updated 2 years ago
- 📚LeetCUDA: Modern CUDA Learn Notes with PyTorch for Beginners🐑, 200+ CUDA Kernels, Tensor Cores, HGEMM, FA-2 MMA.🎉☆10,865Updated this week
- ☆16Jan 14, 2025Updated last year
- ☆11May 2, 2023Updated 3 years ago
- ☆11May 19, 2025Updated 11 months ago
- Minimalist vLLM implementation in Rust☆193May 1, 2026Updated last week