Alternatives and similar repositories for flash-bidirectional-linear-attention:

Users that are interested in flash-bidirectional-linear-attention are comparing it to the libraries listed below

• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆12Updated this week
• BBuf / flash-rwkv
  ☆30Updated 7 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆52Updated 4 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆58Updated 8 months ago
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆28Updated 6 months ago
• glassroom / heinsen_attention
  Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
  ☆24Updated 7 months ago
• NVlabs / GatedDeltaNet
  Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule
  ☆62Updated last week
• thunlp / SparsingLaw
  The open-source materials for paper "Sparsing Law: Towards Large Language Models with Greater Activation Sparsity".
  ☆18Updated last month
• RWKV / RWKV-LM
  RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best…
  ☆21Updated 9 months ago
• habanero-lab / APPy
  APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…
  ☆21Updated last week
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆60Updated 8 months ago
• NVlabs / COAT
  ☆48Updated last week
• Aleph-Alpha / NeurIPS-WANT-submission-efficient-parallelization-layouts
  ☆22Updated last year
• Qualcomm-AI-research / lr-qat
  ☆25Updated 2 months ago
• zyxxmu / cam
  Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference
  ☆29Updated 6 months ago
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆23Updated 6 months ago
• yuzhenmao / IceFormer
  Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).
  ☆21Updated 6 months ago
• SqueezeAILab / SqueezedAttention
  SQUEEZED ATTENTION: Accelerating Long Prompt LLM Inference
  ☆35Updated last month
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆87Updated 8 months ago
• berlino / gated_linear_attention
  ☆99Updated 10 months ago
• dame-cell / Triformer
  Transformers components but in Triton
  ☆29Updated last month
• metacarbon / shareAtt
  Beyond KV Caching: Shared Attention for Efficient LLMs
  ☆13Updated 5 months ago
• pprp / Pruner-Zero
  Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆76Updated last month
• sanagno / adaptively_sparse_attention
  ☆18Updated last year
• sail-sg / SimLayerKV
  The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.
  ☆42Updated 2 months ago
• hahnyuan / ASVD4LLM
  Activation-aware Singular Value Decomposition for Compressing Large Language Models
  ☆55Updated 2 months ago
• MikaStars39 / StableMask
  PyTorch implementation of StableMask (ICML'24)
  ☆12Updated 6 months ago
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆33Updated 3 months ago
• shaochenze / PatchTrain
  Code for paper "Patch-Level Training for Large Language Models"
  ☆74Updated last month