harleyszhang / lite_llama
A light llama-like llm inference framework based on the triton kernel.
☆108Updated last week
Alternatives and similar repositories for lite_llama:
Users that are interested in lite_llama are comparing it to the libraries listed below
- llm theoretical performance analysis tools and support params, flops, memory and latency analysis.☆85Updated 3 months ago
- ☆19Updated last month
- learning how CUDA works☆238Updated last month
- 使用 CUDA C++ 实现的 llama 模型推理框架☆49Updated 5 months ago
- 校招、秋招、春招、实习好项目,带你从零动手实现支持LLama2/3和Qwen2.5的大模型推理框架。☆331Updated 3 weeks ago
- CUDA 算子手撕与面试指南☆314Updated 3 months ago
- ☆122Updated last year
- b站上的课程☆74Updated last year
- A CUDA tutorial to make people learn CUDA program from 0☆225Updated 9 months ago
- A tutorial for CUDA&PyTorch☆132Updated 3 months ago
- Examples of CUDA implementations by Cutlass CuTe☆159Updated 2 months ago
- ☆58Updated 5 months ago
- A minimalist and extensible PyTorch extension for implementing custom backend operators in PyTorch.☆33Updated last year
- ☆148Updated 3 months ago
- ☆264Updated 6 months ago
- 分层解耦的深度学习推理引擎☆72Updated 2 months ago
- Implement custom operators in PyTorch with cuda/c++☆58Updated 2 years ago
- how to learn PyTorch and OneFlow☆425Updated last year
- ☆139Updated last year
- EasyNN是一个面向教学而开发的神经网络推理框架,旨在让大家0 基础也能自主完成推理框架编写!☆27Updated 7 months ago
- 📚FFPA(Split-D): Yet another Faster Flash Attention with O(1) GPU SRAM complexity large headdim, 1.8x~3x↑🎉 faster than SDPA EA.☆169Updated 2 weeks ago
- Implement Flash Attention using Cute.☆76Updated 4 months ago
- 先进编译实验室的个人主页☆66Updated 3 months ago
- ☆115Updated 4 months ago
- easy cuda code☆70Updated 4 months ago
- Tutorials for writing high-performance GPU operators in AI frameworks.☆130Updated last year
- ☆21Updated last month
- Triton Documentation in Chinese Simplified / Triton 中文文档☆66Updated last week
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆46Updated last year
- Optimize softmax in triton in many cases☆20Updated 7 months ago