HanGuo97 / log-linear-attentionLinks
☆216Updated 3 weeks ago
Alternatives and similar repositories for log-linear-attention
Users that are interested in log-linear-attention are comparing it to the libraries listed below
Sorting:
- XAttention: Block Sparse Attention with Antidiagonal Scoring☆166Updated this week
- ☆76Updated 4 months ago
- Efficient triton implementation of Native Sparse Attention.☆168Updated last month
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆131Updated last week
- 🔥 A minimal training framework for scaling FLA models☆178Updated 2 weeks ago
- [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"☆108Updated last month
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆72Updated this week
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆113Updated last month
- Fast and memory-efficient exact attention☆68Updated 3 months ago
- ☆114Updated 3 weeks ago
- Normalized Transformer (nGPT)☆184Updated 7 months ago
- The evaluation framework for training-free sparse attention in LLMs☆69Updated last week
- A sparse attention kernel supporting mix sparse patterns☆238Updated 4 months ago
- 🎬 3.7× faster video generation E2E 🖼️ 1.6× faster image generation E2E ⚡ ColumnSparseAttn 9.3× vs FlashAttn‑3 💨 ColumnSparseGEMM 2.5× …☆74Updated last week
- ☆84Updated last month
- [ICLR 2025] Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule☆174Updated 3 months ago
- Triton implementation of FlashAttention2 that adds Custom Masks.☆120Updated 10 months ago
- Official implementation of "Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding"☆233Updated 2 weeks ago
- [ICML 2025] Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization☆72Updated 3 weeks ago
- Triton implement of bi-directional (non-causal) linear attention☆50Updated 4 months ago
- Patch convolution to avoid large GPU memory usage of Conv2D☆88Updated 5 months ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆126Updated 6 months ago
- Stick-breaking attention☆57Updated last week
- Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".☆127Updated this week
- ☆36Updated 2 months ago
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆41Updated last month
- ☆56Updated 3 months ago
- FlashRNN - Fast RNN Kernels with I/O Awareness☆91Updated 2 weeks ago
- ☆21Updated 3 months ago
- [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.☆81Updated 6 months ago