LDLINGLINGLING / nano_vllm_noteLinks
注释的nano_vllm仓库,并且完成了MiniCPM4的适配以及注册新模型的功能
☆77Updated last month
Alternatives and similar repositories for nano_vllm_note
Users that are interested in nano_vllm_note are comparing it to the libraries listed below
Sorting:
- llm theoretical performance analysis tools and support params, flops, memory and latency analysis.☆108Updated 2 months ago
- 使用 CUDA C++ 实现的 llama 模型推理框架☆62Updated 10 months ago
- Implement Flash Attention using Cute.☆95Updated 9 months ago
- ☆106Updated 4 months ago
- A light llama-like llm inference framework based on the triton kernel.☆153Updated 2 weeks ago
- Triton Documentation in Chinese Simplified / Triton 中文文档☆84Updated 5 months ago
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆267Updated 3 months ago
- ☆147Updated 7 months ago
- Summary of some awesome work for optimizing LLM inference☆111Updated 4 months ago
- gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling☆41Updated this week
- From Minimal GEMM to Everything☆47Updated this week
- ☆139Updated last year
- ☆144Updated 3 months ago
- [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"☆72Updated 3 months ago
- DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit☆66Updated this week
- 🤖FFPA: Extend FlashAttention-2 with Split-D, ~O(1) SRAM complexity for large headdim, 1.8x~3x↑🎉 vs SDPA EA.☆219Updated last month
- NVSHMEM‑Tutorial: Build a DeepEP‑like GPU Buffer☆130Updated 2 weeks ago
- LLM Inference with Deep Learning Accelerator.☆52Updated 8 months ago
- High performance Transformer implementation in C++.☆134Updated 8 months ago
- Triton adapter for Ascend. Mirror of https://gitee.com/ascend/triton-ascend☆75Updated last week
- ⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.☆118Updated 4 months ago
- ☆42Updated last year
- A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆60Updated last month
- Examples of CUDA implementations by Cutlass CuTe☆236Updated 3 months ago
- Optimize softmax in triton in many cases☆21Updated last year
- A PyTorch-like deep learning framework. Just for fun.☆155Updated last year
- A lightweight design for computation-communication overlap.☆177Updated 2 weeks ago
- Codes & examples for "CUDA - From Correctness to Performance"☆114Updated 11 months ago
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆58Updated 6 months ago
- Optimize GEMM with tensorcore step by step☆32Updated last year