注释的nano_vllm仓库，并且完成了MiniCPM4的适配以及注册新模型的功能
☆197Aug 11, 2025Updated 10 months ago
Alternatives and similar repositories for nano_vllm_note
Users that are interested in nano_vllm_note are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An easy-use coroutine lib implement by C++ coroutine and liburing☆32Jun 6, 2025Updated last year
- some hpc project for learning☆28Aug 28, 2024Updated last year
- This project aims to provide a high effective KV cache manage framework for llm inference and improve memory utilization and inference sp…☆61Apr 24, 2026Updated 2 months ago
- ☆13Jan 7, 2025Updated last year
- Code for paper: "Executing Arithmetic: Fine-Tuning Large Language Models as Turing Machines"☆11Oct 11, 2024Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆40Updated this week
- FlashTile is a CUDA Tile IR compiler that is compatible with NVIDIA's tileiras, targeting SM70 through SM121 NVIDIA GPUs.☆61Feb 6, 2026Updated 5 months ago
- Implementation and optimization of matrix multiplication on single CPU (HPC-THU-2023-Autumn)☆18Feb 27, 2024Updated 2 years ago
- A highly efficient library for GEMM operations on Sunway TaihuLight☆18Sep 7, 2020Updated 5 years ago
- A light llama-like llm inference framework based on the triton kernel.☆188Jan 5, 2026Updated 6 months ago
- EasyNN是一个面向教学而开发的神经网络推理框架，旨在让大家0基础也能自主完成推理框架编写！☆40Aug 26, 2024Updated last year
- ☆23Aug 20, 2025Updated 10 months ago
- Nano vLLM☆13Jun 26, 2025Updated last year
- ☆279Jun 9, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- An onnx-based quantitation tool.☆71Jan 8, 2024Updated 2 years ago
- Multiscale Voxelization Operator CUDA Implementation for LiDAR Point Cloud☆21Jul 19, 2023Updated 2 years ago
- A GPU FP32 computation method with Tensor Cores.☆27Dec 8, 2025Updated 7 months ago
- Implement Flash Attention using Cute.☆108Dec 17, 2024Updated last year
- Optimize softmax in triton in many cases☆24Sep 6, 2024Updated last year
- 本仓库在OpenVINO推理框架下部署Nanodet检测算法，并重写预处理和后处理部分，具有超高性能！让你在Intel CPU平台上的检测速度起飞！ 并基于NNCF和PPQ工具将模型量化(PTQ)至int8精度，推理速度更快！☆16Jun 14, 2023Updated 3 years ago
- Matrix multiplication on GPUs for matrices stored on a CPU. Similar to cublasXt, but ported to both NVIDIA and AMD GPUs.☆33Apr 2, 2025Updated last year
- ☆32Jul 2, 2025Updated last year
- 晚上下班不刷手机，学点什么。系列一：CUDA 计算框架 CUFX (Cuda Framework eXtended)。☆17Dec 15, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- patches for huggingface transformers to save memory☆36May 9, 2026Updated 2 months ago
- A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.☆4,520May 17, 2026Updated last month
- A Triton-only attention backend for vLLM☆27Mar 17, 2026Updated 3 months ago
- My submission for the GPUMODE/AMD fp8 mm challenge☆29Jun 4, 2025Updated last year
- Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding☆114Dec 2, 2025Updated 7 months ago
- A parser for PTX 6.5☆13Jun 19, 2023Updated 3 years ago
- Nano vLLM☆14,355Apr 26, 2026Updated 2 months ago
- ☆45Apr 9, 2024Updated 2 years ago
- ☆110May 31, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- study of cutlass☆22Nov 10, 2024Updated last year
- HPC Game Platform☆11Apr 20, 2023Updated 3 years ago
- a highly-efficient library for deep neural networks based on Sunway TaihuLight supercomputer.☆17Sep 3, 2018Updated 7 years ago
- ☆121May 16, 2025Updated last year
- ☆49Apr 15, 2024Updated 2 years ago
- 📚LeetCUDA: Modern CUDA Learn Notes with PyTorch for Beginners🐑, 200+ CUDA Kernels, Tensor Cores, HGEMM, FA-2 MMA.🎉☆11,461Updated this week
- ☆11Jun 9, 2023Updated 3 years ago