Alternatives and similar repositories for flash_attn_gqa

Users that are interested in flash_attn_gqa are comparing it to the libraries listed below

• yikangshen / megablocks
  ☆20Updated last year
• kyegomez / Reka-Torch
  Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch
  ☆30Updated this week
• ernie-research / Tool-Augmented-Reward-Model
  [ICLR'24 spotlight] Tool-Augmented Reward Modeling
  ☆50Updated 2 weeks ago
• qiuzh20 / gated_attention
  The official implementation for Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
  ☆44Updated last month
• iiis-ai / IterativeQuestionComposing
  Official implementation of AAAI 2025 paper "Augmenting Math Word Problems via Iterative Question Composing"(https://arxiv.org/abs/2401.09…
  ☆20Updated 6 months ago
• RUCAIBox / BAMBOO
  ☆35Updated last year
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆34Updated 9 months ago
• jungokasai / T2R
  ☆14Updated 2 years ago
• yuzhaouoe / pretraining-data-packing
  [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training
  ☆22Updated 10 months ago
• sustcsonglin / disco-pointer
  Official Implementation of ACL2023: Don't Parse, Choose Spans! Continuous and Discontinuous Constituency Parsing via Autoregressive Span …
  ☆13Updated last year
• yiqingxyq / RepoST
  Code for "RepoST: Scalable Repository-Level Coding Environment Construction with Sandbox Testing"
  ☆22Updated 3 months ago
• juvi21 / CoPE-cuda
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆22Updated last year
• ldery / Bonsai
  Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"
  ☆28Updated last year
• chijames / KERPLE
  ☆19Updated 2 years ago
• LuLuLuyi / LongHeads
  LongHeads: Multi-Head Attention is Secretly a Long Context Processor
  ☆29Updated last year
• whyNLP / Probabilistic-Transformer
  A probabilitic model for contextual word representation. Accepted to ACL2023 Findings.
  ☆23Updated last year
• Hunter-DDM / stablemoe
  Code for the ACL-2022 paper "StableMoE: Stable Routing Strategy for Mixture of Experts"
  ☆47Updated 2 years ago
• GSYfate / knnlm-limits
  Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"
  ☆23Updated last month
• DRSY / KV_Compression
  [EMNLP 2023]Context Compression for Auto-regressive Transformers with Sentinel Tokens
  ☆24Updated last year
• yuh-zha / Align
  Align, a general text alignment function
  ☆15Updated last year
• kaistAI / factual-knowledge-acquisition
  ☆19Updated last month
• RenzeLou / Muffin
  MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction-Following
  ☆16Updated 7 months ago
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆57Updated 9 months ago
• SkyworkAI / MindLink
  ☆46Updated this week
• OpenLMLab / ParallelTokenizer
  Use the tokenizer in parallel to achieve superior acceleration
  ☆16Updated last year
• HazyResearch / prefix-linear-attention
  ☆55Updated 11 months ago
• zhichaoxu-shufe / context-aware-decoding-qfs
  ☆12Updated last year
• uservan / speculative_thinking
  ☆20Updated 3 weeks ago
• sail-sg / SkyLadder
  The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling
  ☆32Updated 3 months ago
• ChengpengLi1003 / DotaMath
  ☆30Updated 5 months ago