注释的nano_vllm仓库，并且完成了MiniCPM4的适配以及注册新模型的功能
☆184Aug 11, 2025Updated 9 months ago
Alternatives and similar repositories for nano_vllm_note
Users that are interested in nano_vllm_note are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An easy-use coroutine lib implement by C++ coroutine and liburing☆32Jun 6, 2025Updated 11 months ago
- some hpc project for learning☆28Aug 28, 2024Updated last year
- 该仓库主要记录 大模型（LLMs） 算法工程师相关的面试题与我写的答案☆27Oct 14, 2023Updated 2 years ago
- YOLOv8 🚀 in PyTorch > ONNX > CoreML > TFLite☆20Feb 9, 2026Updated 3 months ago
- This project aims to provide a high effective KV cache manage framework for llm inference and improve memory utilization and inference sp…☆56Apr 24, 2026Updated last month
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆13Jan 7, 2025Updated last year
- Code for paper: "Executing Arithmetic: Fine-Tuning Large Language Models as Turing Machines"☆11Oct 11, 2024Updated last year
- ☆29May 22, 2026Updated last week
- FlashTile is a CUDA Tile IR compiler that is compatible with NVIDIA's tileiras, targeting SM70 through SM121 NVIDIA GPUs.☆60Feb 6, 2026Updated 3 months ago
- Implementation and optimization of matrix multiplication on single CPU (HPC-THU-2023-Autumn)☆18Feb 27, 2024Updated 2 years ago
- A highly efficient library for GEMM operations on Sunway TaihuLight☆18Sep 7, 2020Updated 5 years ago
- EasyNN是一个面向教学而开发的神经网络推理框架，旨在让大家0基础也能自主完成推理框架编写！☆39Aug 26, 2024Updated last year
- ☆23Aug 20, 2025Updated 9 months ago
- Nano vLLM☆13Jun 26, 2025Updated 11 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- An onnx-based quantitation tool.☆71Jan 8, 2024Updated 2 years ago
- ToyLLM: Learning LLM from Scratch☆25May 18, 2026Updated last week
- A GPU FP32 computation method with Tensor Cores.☆27Dec 8, 2025Updated 5 months ago
- Implement Flash Attention using Cute.☆108Dec 17, 2024Updated last year
- Optimize softmax in triton in many cases☆24Sep 6, 2024Updated last year
- 跟着Tensorrt_pro学习各种知识☆39Nov 25, 2022Updated 3 years ago
- 本仓库在OpenVINO推理框架下部署Nanodet检测算法，并重写预处理和后处理部分，具有超高性能！让你在Intel CPU平台上的检测速度起飞！ 并基于NNCF和PPQ工具将模型量化(PTQ)至int8精度，推理速度更快！☆16Jun 14, 2023Updated 2 years ago
- Matrix multiplication on GPUs for matrices stored on a CPU. Similar to cublasXt, but ported to both NVIDIA and AMD GPUs.☆32Apr 2, 2025Updated last year
- ☆32Jul 2, 2025Updated 10 months ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- 晚上下班不刷手机，学点什么。系列一：CUDA 计算框架 CUFX (Cuda Framework eXtended)。☆16Dec 15, 2024Updated last year
- patches for huggingface transformers to save memory☆36May 9, 2026Updated 2 weeks ago
- A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.☆4,255May 17, 2026Updated last week
- A Triton-only attention backend for vLLM☆25Mar 17, 2026Updated 2 months ago
- My submission for the GPUMODE/AMD fp8 mm challenge☆29Jun 4, 2025Updated 11 months ago
- Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding☆105Dec 2, 2025Updated 5 months ago
- Nano vLLM☆13,595Apr 26, 2026Updated last month
- ☆105May 31, 2025Updated 11 months ago
- study of cutlass☆22Nov 10, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆121May 16, 2025Updated last year
- A Toolkit for Fine-Tuning Large Language Models with LoRA and DeepSpeed☆11Apr 14, 2023Updated 3 years ago
- flash attention 优化日志☆29Jun 4, 2025Updated 11 months ago
- ☆49Apr 15, 2024Updated 2 years ago
- ☆106Updated this week
- ☆11Jun 9, 2023Updated 2 years ago
- ☆16Jan 14, 2025Updated last year