FlagOpen / FlagAttentionLinks
A collection of memory efficient attention operators implemented in the Triton language.
☆275Updated last year
Alternatives and similar repositories for FlagAttention
Users that are interested in FlagAttention are comparing it to the libraries listed below
Sorting:
- PyTorch bindings for CUTLASS grouped GEMM.☆134Updated 3 weeks ago
- flash attention tutorial written in python, triton, cuda, cutlass☆402Updated 2 months ago
- Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.☆199Updated this week
- ☆228Updated last year
- This repository contains integer operators on GPUs for PyTorch.☆211Updated last year
- An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).☆260Updated 3 weeks ago
- ☆96Updated 11 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆107Updated 2 months ago
- FlagGems is an operator library for large language models implemented in the Triton Language.☆640Updated this week
- A Quirky Assortment of CuTe Kernels☆388Updated this week
- Zero Bubble Pipeline Parallelism☆415Updated 3 months ago
- ☆145Updated 5 months ago
- ⚡️FFPA: Extend FlashAttention-2 with Split-D, achieve ~O(1) SRAM complexity for large headdim, 1.8x~3x↑ vs SDPA.🎉☆194Updated 2 months ago
- Applied AI experiments and examples for PyTorch☆289Updated 2 months ago
- A Easy-to-understand TensorOp Matmul Tutorial☆370Updated 10 months ago
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆39Updated 5 months ago
- Pipeline Parallelism Emulation and Visualization☆54Updated last month
- Dynamic Memory Management for Serving LLMs without PagedAttention☆405Updated 2 months ago
- [MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving☆318Updated last year
- An easy-to-use package for implementing SmoothQuant for LLMs☆103Updated 4 months ago
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.☆281Updated this week
- [USENIX ATC '24] Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Paral…☆61Updated last year
- ☆139Updated last year
- QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.☆137Updated 4 months ago
- Puzzles for learning Triton, play it with minimal environment configuration!☆446Updated 8 months ago
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity☆216Updated last year
- ☆92Updated 4 months ago
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆212Updated 11 months ago
- ☆207Updated 8 months ago
- ☆331Updated last year