Alternatives and similar repositories for flash-attn-101

Users that are interested in flash-attn-101 are comparing it to the libraries listed below

• kpup1710 / CAMEx
  [ICLR 2025] CAMEx: Curvature-Aware Merging of Experts
  ☆22Updated 6 months ago
• baochi0212 / LaVy
  Pioneering in Vietnamese Multimodal Large Language Model
  ☆51Updated 7 months ago
• indri-voice / vit.triton
  VIT inference in triton because, why not?
  ☆31Updated last year
• ZaloAI-Jaist / VMLU
  ☆71Updated last year
• lucidrains / PEER-pytorch
  Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind
  ☆127Updated last year
• fal-ai-community / nano-mdm
  Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun
  ☆56Updated 5 months ago
• PiotrNawrot / sparse-frontier
  The evaluation framework for training-free sparse attention in LLMs
  ☆91Updated 2 months ago
• Fsoft-AIC / LibMoE
  LibMoE: A LIBRARY FOR COMPREHENSIVE BENCHMARKING MIXTURE OF EXPERTS IN LARGE LANGUAGE MODELS
  ☆40Updated 2 months ago
• nil0x9 / flash-muon
  Flash-Muon: An Efficient Implementation of Muon Optimizer
  ☆174Updated 2 months ago
• CASE-Lab-UMD / LLM-Drop
  The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".
  ☆174Updated 5 months ago
• HanGuo97 / log-linear-attention
  ☆240Updated 2 months ago
• andravin / spio
  Memory-Efficient CUDA kernels for training ConvNets with PyTorch.
  ☆42Updated 6 months ago
• duongna21 / bartflax
  Pre-training script for BART in JAX/Flax
  ☆38Updated 3 years ago
• mengxiayu / LLMSuperWeight
  Code for studying the super weight in LLM
  ☆115Updated 8 months ago
• cloneofsimo / min-fsdp
  ☆87Updated last year
• kuleshov-group / remdm
  Remasking Discrete Diffusion Models with Inference-Time Scaling
  ☆38Updated 5 months ago
• lucidrains / maskbit-pytorch
  Implementation of the proposed MaskBit from Bytedance AI
  ☆82Updated 9 months ago
• sustcsonglin / linear-attention-and-beyond-slides
  ☆80Updated 6 months ago
• zaydzuhri / softpick-attention
  Implementations of attention with the softpick function, naive and FlashAttention-2
  ☆83Updated 4 months ago
• sayakpaul / simple-image-recaptioning
  Recaption large (Web)Datasets with vllm and save the artifacts.
  ☆52Updated 9 months ago
• cmu-flame / FLAME-MoE
  Official repository for FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models
  ☆23Updated last month
• IST-DASLab / DarwinLM
  Official Pytorch Implementation of Paper "DarwinLM: Evolutionary Structured Pruning of Large Language Models"
  ☆17Updated 6 months ago
• TianjinYellow / SPAM-Optimizer
  ☆34Updated 5 months ago
• ShiZhengyan / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆39Updated last year
• zhixuan-lin / forgetting-transformer
  [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning
  ☆124Updated 2 weeks ago
• samsja / muon_fsdp_2
  Muon fsdp 2
  ☆42Updated 3 weeks ago
• nbasyl / DoRA
  Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
  ☆124Updated last year
• tspeterkim / mixed-precision-from-scratch
  Mixed precision training from scratch with Tensors and CUDA
  ☆25Updated last year
• bluorion-com / ZClip
  Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".
  ☆132Updated 3 weeks ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆111Updated 7 months ago