Alternatives and similar repositories for LASP

Users that are interested in LASP are comparing it to the libraries listed below

• li-plus / flash-preference
  Accelerate LLM preference tuning via prefix sharing with a single line of code
  ☆41Updated last month
• feifeibear / Odysseus-Transformer
  Odysseus: Playground of LLM Sequence Parallelism
  ☆69Updated 11 months ago
• HazyResearch / prefix-linear-attention
  ☆54Updated 10 months ago
• xiayuqing0622 / flex_head_fa
  Fast and memory-efficient exact attention
  ☆68Updated 3 months ago
• Infini-AI-Lab / gsm_infinite
  ☆45Updated 3 months ago
• PiotrNawrot / sparse-frontier
  Official implementation of "The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs"
  ☆32Updated last month
• YuchuanTian / DiJiang
  [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…
  ☆101Updated 11 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆58Updated last year
• Dao-AILab / grouped-latent-attention
  ☆53Updated this week
• DRSY / EasyKV
  Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
  ☆63Updated last year
• MayDomine / Burst-Attention
  Distributed IO-aware Attention algorithm
  ☆20Updated 9 months ago
• frankxwang / dpo-prefix-sharing
  DPO, but faster 🚀
  ☆42Updated 5 months ago
• OpenSparseLLMs / Linear-MoE
  ☆93Updated 2 weeks ago
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆27Updated last year
• BBuf / flash-rwkv
  ☆31Updated last year
• XunhaoLai / native-sparse-attention-triton
  Efficient triton implementation of Native Sparse Attention.
  ☆155Updated last week
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆128Updated last year
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆75Updated last year
• berlino / gated_linear_attention
  ☆103Updated last year
• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆146Updated 3 weeks ago
• kyegomez / Blockwise-Parallel-Transformer
  32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.
  ☆48Updated last year
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆43Updated 8 months ago
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆39Updated last year
• dame-cell / Triformer
  Transformers components but in Triton
  ☆33Updated 3 weeks ago
• hdong920 / GRIFFIN
  ☆37Updated 9 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆121Updated 4 months ago
• tridao / flash-attention-wheels
  ☆49Updated last year
• PiotrNawrot / nano-sparse-attention
  The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.
  ☆62Updated 4 months ago
• qiuzh20 / gated_attention
  The official implementation for Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
  ☆38Updated 3 weeks ago
• sustcsonglin / linear-attention-and-beyond-slides
  ☆74Updated 3 months ago