注释的nano_vllm仓库，并且完成了MiniCPM4的适配以及注册新模型的功能
☆168Aug 11, 2025Updated 6 months ago
Alternatives and similar repositories for nano_vllm_note
Users that are interested in nano_vllm_note are comparing it to the libraries listed below
Sorting:
- Persistent dense gemm for Hopper in `CuTeDSL`☆15Aug 9, 2025Updated 7 months ago
- ☆13Jan 7, 2025Updated last year
- FlashTile is a CUDA Tile IR compiler that is compatible with NVIDIA's tileiras, targeting SM70 through SM121 NVIDIA GPUs.☆56Feb 6, 2026Updated last month
- EasyNN是一个面向教学而开发的神经网络推理框架，旨在让大家0基础也能自主完成推理框架编写！☆39Aug 26, 2024Updated last year
- A Triton-only attention backend for vLLM☆24Feb 11, 2026Updated 3 weeks ago
- ☆20Aug 20, 2025Updated 6 months ago
- 该仓库主要记录 大模型（LLMs） 算法工程师相关的面试题与我写的答案☆27Oct 14, 2023Updated 2 years ago
- ☆39Apr 9, 2024Updated last year
- A light llama-like llm inference framework based on the triton kernel.☆172Jan 5, 2026Updated 2 months ago
- ☆32Jul 2, 2025Updated 8 months ago
- Implementation and optimization of matrix multiplication on single CPU (HPC-THU-2023-Autumn)☆18Feb 27, 2024Updated 2 years ago
- 本仓库在OpenVINO推理框架下部署Nanodet检测算法，并重写预处理和后处理部分，具有超高性能！让你在Intel CPU平台上的检测速度起飞！ 并基于NNCF和PPQ工具将模型量化(PTQ)至int8精度，推理速度更快！☆16Jun 14, 2023Updated 2 years ago
- An onnx-based quantitation tool.☆70Jan 8, 2024Updated 2 years ago
- 晚上下班不刷手机，学点什么。系列一：CUDA 计算框架 CUFX (Cuda Framework eXtended)。☆16Dec 15, 2024Updated last year
- 跟着Tensorrt_pro学习各种知识☆40Nov 25, 2022Updated 3 years ago
- study of cutlass☆22Nov 10, 2024Updated last year
- Implement Flash Attention using Cute.☆102Dec 17, 2024Updated last year
- ☆45Feb 27, 2026Updated last week
- Sample Codes using NVSHMEM on Multi-GPU☆30Jan 22, 2023Updated 3 years ago
- FireQ: Fast INT4-FP8 Kernel and RoPE-aware Quantization for LLM Inference Acceleration☆20Jun 27, 2025Updated 8 months ago
- ☆116May 16, 2025Updated 9 months ago
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆60Mar 25, 2025Updated 11 months ago
- ☆88May 31, 2025Updated 9 months ago
- Optimize softmax in triton in many cases☆23Sep 6, 2024Updated last year
- ☆65Apr 26, 2025Updated 10 months ago
- Gensis is a lightweight deep learning framework written from scratch in Python, with Triton as its backend for high-performance computing…☆37Jan 15, 2026Updated last month
- some hpc project for learning☆26Aug 28, 2024Updated last year
- Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.☆73Sep 8, 2024Updated last year
- Matrix multiplication on GPUs for matrices stored on a CPU. Similar to cublasXt, but ported to both NVIDIA and AMD GPUs.☆32Apr 2, 2025Updated 11 months ago
- from MHA, MQA, GQA to MLA by 苏剑林, with code☆43Feb 19, 2025Updated last year
- ☆33Jul 8, 2025Updated 8 months ago
- An MLIR-based compiler that takes GPU kernels and compiles them to real hardware instructions. Interactive web visualizer included.☆109Feb 24, 2026Updated last week
- Vstream - Video Analytics pipeline with Hardware based accelerations (dev - stage)☆10Feb 2, 2024Updated 2 years ago
- 基于天池新闻推荐赛数据集实现的新闻推荐☆43Dec 17, 2024Updated last year
- ☆31Aug 25, 2023Updated 2 years ago
- ☆40Updated this week
- 针对建筑规范文本数据的知识图谱实体关系提取，知识图谱构建，检索增强生成DEMO☆37Aug 7, 2024Updated last year
- ☆52Jan 5, 2026Updated 2 months ago
- patches for huggingface transformers to save memory☆35Jun 2, 2025Updated 9 months ago