Alternatives and similar repositories for mla-experiments:

Users that are interested in mla-experiments are comparing it to the libraries listed below

• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆82Updated this week
• sustcsonglin / linear-attention-and-beyond-slides
  ☆63Updated last month
• PiotrNawrot / nano-sparse-attention
  The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.
  ☆58Updated 2 months ago
• xiayuqing0622 / flex_head_fa
  Fast and memory-efficient exact attention
  ☆67Updated 3 weeks ago
• OpenSparseLLMs / Linear-MoE
  ☆70Updated 2 weeks ago
• dame-cell / Triformer
  Transformers components but in Triton
  ☆32Updated last week
• XunhaoLai / native-sparse-attention-triton
  Efficient triton implementation of Native Sparse Attention.
  ☆116Updated this week
• OpenNLPLab / LASP
  Linear Attention Sequence Parallelism (LASP)
  ☆79Updated 9 months ago
• HanGuo97 / lq-lora
  ☆125Updated last year
• SqueezeAILab / SqueezedAttention
  SQUEEZED ATTENTION: Accelerating Long Prompt LLM Inference
  ☆45Updated 4 months ago
• bytedance / FlexPrefill
  Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
  ☆62Updated last week
• AnswerDotAI / cold-compress
  Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…
  ☆122Updated 7 months ago
• lucidrains / coconut-pytorch
  Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch
  ☆161Updated 2 months ago
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆154Updated 9 months ago
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆125Updated last year
• epfml / dynamic-sparse-flash-attention
  ☆141Updated last year
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆86Updated 2 weeks ago
• thu-ml / ReMoE
  [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.
  ☆65Updated 3 months ago
• RobertCsordas / moeut
  ☆74Updated 7 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆60Updated 11 months ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆110Updated 2 months ago
• srush / mamba-primer
  ☆37Updated 11 months ago
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆49Updated 2 weeks ago
• mgmalek / efficient_cross_entropy
  ☆95Updated 10 months ago
• krafton-ai / mambaformer-icl
  MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248
  ☆51Updated 9 months ago
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆34Updated 9 months ago
• huggingface / kernels
  Load compute kernels from the Hub
  ☆99Updated this week
• whyNLP / LCKV
  Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…
  ☆148Updated 2 months ago
• gpu-mode / ring-attention
  ring-attention experiments
  ☆128Updated 5 months ago
• softmax1 / Flash-Attention-Softmax-N
  CUDA and Triton implementations of Flash Attention with SoftmaxN.
  ☆68Updated 10 months ago