vllm-project / flash-attentionLinks
Fast and memory-efficient exact attention
☆82Updated this week
Alternatives and similar repositories for flash-attention
Users that are interested in flash-attention are comparing it to the libraries listed below
Sorting:
- ☆92Updated 4 months ago
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆110Updated last year
- A lightweight design for computation-communication overlap.☆154Updated last month
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆103Updated 2 months ago
- An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).☆260Updated 2 weeks ago
- ☆145Updated 4 months ago
- ☆96Updated 10 months ago
- DeeperGEMM: crazy optimized version☆70Updated 2 months ago
- ⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.☆91Updated 2 months ago
- AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…☆254Updated last month
- Standalone Flash Attention v2 kernel without libtorch dependency☆111Updated 10 months ago
- ⚡️FFPA: Extend FlashAttention-2 with Split-D, achieve ~O(1) SRAM complexity for large headdim, 1.8x~3x↑ vs SDPA.🎉☆194Updated 2 months ago
- Pipeline Parallelism Emulation and Visualization☆54Updated last month
- Dynamic Memory Management for Serving LLMs without PagedAttention☆405Updated 2 months ago
- ☆60Updated 3 months ago
- Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.☆193Updated this week
- QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.☆135Updated 3 months ago
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity☆216Updated last year
- High performance Transformer implementation in C++.☆128Updated 6 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆106Updated 2 months ago
- We invite you to visit and follow our new repository at https://github.com/microsoft/TileFusion. TiledCUDA is a highly efficient kernel …☆183Updated 6 months ago
- ☆85Updated 8 months ago
- A low-latency & high-throughput serving engine for LLMs☆397Updated 2 months ago
- A collection of memory efficient attention operators implemented in the Triton language.☆273Updated last year
- Benchmark code for the "Online normalizer calculation for softmax" paper☆95Updated 7 years ago
- Perplexity GPU Kernels☆413Updated 2 weeks ago
- PyTorch library for cost-effective, fast and easy serving of MoE models.☆215Updated 3 weeks ago
- ☆195Updated 2 months ago
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆39Updated 5 months ago
- ☆80Updated last week