Flash Attention in 300-500 lines of CUDA/C++
☆39Aug 22, 2025Updated last year
Alternatives and similar repositories for flash-attention-minimal
Users that are interested in flash-attention-minimal are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Fast, Numerically Stable, and Auto-Differentiable Spectral Clipping via Newton-Schulz Iteration☆12Jun 21, 2025Updated last year
- ☆48Nov 1, 2025Updated last year
- Distillation pipeline from pretrained Transformers to customized FLA models☆13Aug 4, 2025Updated last year
- Comparative study and experimentation on standard vs mHC vs attention residual (full and block)☆13Mar 17, 2026Updated 7 months ago
- Code to generate figures of paper "When do spectral gradient updates help in deep learning?"☆19Dec 3, 2025Updated 11 months ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Reduce kernel based on CUTLASS CuTe and TMA.☆12Sep 25, 2025Updated last year
- Flash Attention in ~100 lines of CUDA (forward pass only)☆1,203Dec 30, 2024Updated last year
- codes and plots for "Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs"☆11Dec 30, 2024Updated last year
- FlexAttention w/ FlashAttention3 Support☆27Oct 5, 2024Updated 2 years ago
- Dynamic analysis of multithreaded C programs☆13Feb 7, 2020Updated 6 years ago
- The official implementation of HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization☆19Mar 7, 2025Updated last year
- [COLM'25] A Controlled Study on Long Context Extension and Generalization in LLMs☆65Mar 9, 2026Updated 7 months ago
- This repositories contains the reference implementation for the Sparse Delta Memory paper.More precisely, it contains the model definitio…☆43Oct 24, 2026Updated last week
- Stick-breaking attention☆64Jul 1, 2025Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Latent Large Language Models☆19Aug 24, 2024Updated 2 years ago
- restore tone for missing tone sentences☆13Jul 29, 2019Updated 7 years ago
- Personal solutions to the Triton Puzzles☆22Jul 18, 2024Updated 2 years ago
- Flash-Linear-Attention models beyond language☆21Aug 28, 2025Updated last year
- Efficient Finetuning for OpenAI GPT-OSS☆24Oct 2, 2025Updated last year
- ☆22Oct 3, 2024Updated 2 years ago
- 使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention☆82Aug 12, 2024Updated 2 years ago
- ☆38Dec 31, 2025Updated 10 months ago
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆27Apr 17, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- A multi-agent parallel inference orchestrator for MarketAgents☆12Mar 27, 2025Updated last year
- Transformers components but in Triton☆35May 9, 2025Updated last year
- Final research artifacts from Épi: 106 improved constructions, 15 mathematical results, and their certificates.☆12Sep 14, 2026Updated last month
- ☆23Dec 1, 2021Updated 4 years ago
- ☆29Jun 10, 2025Updated last year
- Implement Flash Attention using Cute.☆113Dec 17, 2024Updated last year
- Effective transpose on Hopper GPU☆29Sep 6, 2025Updated last year
- A vector field rendering library☆17Jul 31, 2019Updated 7 years ago
- LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models☆79Oct 16, 2024Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆20May 30, 2024Updated 2 years ago
- flash attention tutorial written in python, triton, cuda, cutlass☆540Oct 22, 2026Updated 2 weeks ago
- Simulator for SpecDecode Bench☆14Apr 1, 2026Updated 7 months ago
- triton ver of gqa flash attn, based on the tutorial☆12Aug 4, 2024Updated 2 years ago
- Focused on fast experimentation and simplicity☆77Dec 24, 2024Updated last year
- 手写 Triton kernel 实现的 Qwen3.5-0.8B 纯文本推理引擎（batch=1, BF16, greedy）☆12Sep 10, 2026Updated last month
- HGRN2: Gated Linear RNNs with State Expansion☆59Aug 20, 2024Updated 2 years ago