Alternatives and similar repositories for FlashMHA:

Users that are interested in FlashMHA are comparing it to the libraries listed below

• kyegomez / TinyGPTV
  Simple Implementation of TinyGPTV in super simple Zeta lego blocks
  ☆15Updated 2 months ago
• kyegomez / MobileVLM
  Implementation of the LDP module block in PyTorch and Zeta from the paper: "MobileVLM: A Fast, Strong and Open Vision Language Assistant …
  ☆15Updated 10 months ago
• BBuf / flash-rwkv
  ☆31Updated 7 months ago
• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆35Updated last week
• tridao / flash-attention-wheels
  ☆45Updated last year
• pprp / Pruner-Zero
  Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆76Updated last month
• LiqunMa / FBI-LLM
  FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
  ☆46Updated 6 months ago
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆42Updated 6 months ago
• kyegomez / Paper-Implementation-Template
  A simple reproducible template to implement AI research papers
  ☆22Updated 4 months ago
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆28Updated 7 months ago
• IST-DASLab / QIGen
  Repository for CPU Kernel Generation for LLM Inference
  ☆25Updated last year
• tanaymeh / mamba-train
  A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM
  ☆50Updated 9 months ago
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆79Updated last week
• OpenNLPLab / LASP
  Linear Attention Sequence Parallelism (LASP)
  ☆74Updated 7 months ago
• thepowerfuldeez / OLMo
  My fork os allen AI's OLMo for educational purposes.
  ☆30Updated last month
• hetailang / SqueezeAttention
  ☆37Updated 3 months ago
• IST-DASLab / RoSA
  ☆38Updated 11 months ago
• lucidrains / vit-arc-slot
  Explorations into improving ViTArc with Slot Attention
  ☆37Updated 3 months ago
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆38Updated 10 months ago
• kyegomez / PaLM2-VAdapter
  Implementation of "PaLM2-VAdapter:" from the multi-modal model paper: "PaLM2-VAdapter: Progressively Aligned Language Model Makes a Stron…
  ☆16Updated 2 months ago
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆36Updated last year
• kyegomez / Qwen-VL
  My personal implementation of the model from "Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities", they haven't rel…
  ☆13Updated 11 months ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆107Updated 2 weeks ago
• thu-nics / MoA
  The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>
  ☆110Updated last month
• kiddyboots216 / lottery-ticket-adaptation
  Lottery Ticket Adaptation
  ☆37Updated 2 months ago
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆35Updated 7 months ago
• MDK8888 / vllmini
  A minimal implementation of vllm.
  ☆32Updated 5 months ago
• BlinkDL / LM-Trick-Questions
  Here we collect trick questions and failed tasks for open source LLMs to improve them.
  ☆32Updated last year
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆76Updated 7 months ago
• RAIVNLab / MatFormer-OLMo
  Code repository for the public reproduction of the language modelling experiments on "MatFormer: Nested Transformer for Elastic Inference…
  ☆18Updated last year