Alternatives and similar repositories for FlashAttention-PyTorch:

Users that are interested in FlashAttention-PyTorch are comparing it to the libraries listed below

• madsys-dev / deepseekv2-profile
  ☆73Updated 6 months ago
• kyegomez / FlashAttention20
  Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels
  ☆102Updated last year
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆415Updated last month
• pprp / Awesome-LLM-Quantization
  Awesome list for LLM quantization
  ☆160Updated last month
• OpenNLPLab / lightning-attention
  Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
  ☆249Updated 9 months ago
• FlagOpen / FlagAttention
  A collection of memory efficient attention operators implemented in the Triton language.
  ☆233Updated 7 months ago
• void-main / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆59Updated last year
• kyegomez / FlashAttention20Triton
  Triton implementation of Flash Attention2.0
  ☆29Updated last year
• 66RING / tiny-flash-attention
  flash attention tutorial written in python, triton, cuda, cutlass
  ☆255Updated 3 weeks ago
• xijiu9 / Train_Transformers_with_INT4
  ☆140Updated last year
• DD-DuDa / BitDistiller
  [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.
  ☆98Updated 8 months ago
• zhuzilin / ring-flash-attention
  Ring attention implementation with flash attention
  ☆660Updated last month
• BlinkDL / RWKV-CUDA
  The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )
  ☆217Updated last month
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆125Updated last year
• FasterDecoding / SnapKV
  ☆217Updated 8 months ago
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆152Updated 6 months ago
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆89Updated 8 months ago
• zms1999 / SmartMoE
  A MoE impl for PyTorch, [ATC'23] SmartMoE
  ☆61Updated last year
• Bruce-Lee-LY / flash_attention_inference
  Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.
  ☆34Updated 4 months ago
• SiriusNEO / Triton-Puzzles-Lite
  Puzzles for learning Triton, play it with minimal environment configuration!
  ☆207Updated last month
• Ascend / AscendSpeed
  ☆76Updated last year
• chunhuizhang / pytorch_distribute_tutorials
  pytorch distribute tutorials
  ☆101Updated 3 months ago
• fkodom / grouped-query-attention-pytorch
  (Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from …
  ☆149Updated 8 months ago
• harleyszhang / llm_counts
  llm theoretical performance analysis tools and support params, flops, memory and latency analysis.
  ☆76Updated 3 weeks ago
• HandH1998 / QQQ
  QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.
  ☆95Updated last month
• FasterDecoding / REST
  REST: Retrieval-Based Speculative Decoding, NAACL 2024
  ☆190Updated last month
• hemingkx / Spec-Bench
  Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)
  ☆218Updated 3 months ago
• AlibabaPAI / FLASHNN
  ☆79Updated 4 months ago
• October2001 / Awesome-KV-Cache-Compression
  📰 Must-read papers on KV Cache Compression (constantly updating 🤗).
  ☆276Updated 2 weeks ago
• luliyucoordinate / cute-flash-attention
  Implement Flash Attention using Cute.
  ☆67Updated last month