Alternatives and similar repositories for mla-experiments:

Users that are interested in mla-experiments are comparing it to the libraries listed below

• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆107Updated last week
• mengxiayu / LLMSuperWeight
  Code for studying the super weight in LLM
  ☆98Updated 4 months ago
• dame-cell / Triformer
  Transformers components but in Triton
  ☆32Updated last month
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆90Updated this week
• XunhaoLai / native-sparse-attention-triton
  Efficient triton implementation of Native Sparse Attention.
  ☆136Updated 2 weeks ago
• PiotrNawrot / nano-sparse-attention
  The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.
  ☆59Updated 2 months ago
• HanGuo97 / lq-lora
  ☆125Updated last year
• astramind-ai / Mixture-of-depths
  Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆158Updated 10 months ago
• xiayuqing0622 / flex_head_fa
  Fast and memory-efficient exact attention
  ☆67Updated last month
• mgmalek / efficient_cross_entropy
  ☆100Updated 10 months ago
• frankxwang / dpo-prefix-sharing
  DPO, but faster 🚀
  ☆40Updated 4 months ago
• shadowpa0327 / Palu
  [ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projection
  ☆99Updated 2 months ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆110Updated 3 months ago
• tridao / flash-attention-wheels
  ☆48Updated last year
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆97Updated 6 months ago
• shawntan / scattermoe
  Triton-based implementation of Sparse Mixture of Experts.
  ☆210Updated 4 months ago
• OpenNLPLab / LASP
  Linear Attention Sequence Parallelism (LASP)
  ☆82Updated 10 months ago
• sustcsonglin / linear-attention-and-beyond-slides
  ☆69Updated last month
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆35Updated 10 months ago
• jaymody / speculative-sampling
  Simple implementation of Speculative Sampling in NumPy for GPT-2.
  ☆93Updated last year
• krafton-ai / mambaformer-icl
  MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248
  ☆51Updated 10 months ago
• whyNLP / LCKV
  Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…
  ☆148Updated 2 weeks ago
• fkodom / grouped-query-attention-pytorch
  (Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from …
  ☆162Updated 11 months ago
• CASE-Lab-UMD / Unified-MoE-Compression
  The official implementation of the paper "Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques (TMLR)".
  ☆66Updated last month
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆72Updated 7 months ago
• OpenSparseLLMs / Linear-MoE
  ☆75Updated last week
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆57Updated 6 months ago
• wdlctc / mini-s
  ☆50Updated 5 months ago
• mit-han-lab / x-attention
  XAttention: Block Sparse Attention with Antidiagonal Scoring
  ☆140Updated 3 weeks ago
• Aleph-Alpha / NeurIPS-WANT-submission-efficient-parallelization-layouts
  ☆22Updated last year