kyegomez / FlashMHALinks
An simple pytorch implementation of Flash MultiHead Attention
☆19Updated last year
Alternatives and similar repositories for FlashMHA
Users that are interested in FlashMHA are comparing it to the libraries listed below
Sorting:
- ☆32Updated last year
- Linear Attention Sequence Parallelism (LASP)☆87Updated last year
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆25Updated 2 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆77Updated last year
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆43Updated 3 months ago
- ☆99Updated 4 months ago
- Triton implementation of FlashAttention2 that adds Custom Masks.☆138Updated last year
- ☆89Updated 7 months ago
- [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…☆102Updated last year
- ☆251Updated 4 months ago
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- ☆57Updated last year
- ☆129Updated 4 months ago
- Tiny-FSDP, a minimalistic re-implementation of the PyTorch FSDP☆82Updated last month
- Fast and memory-efficient exact attention☆70Updated 7 months ago
- ☆24Updated 6 months ago
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆48Updated 2 years ago
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆193Updated 3 months ago
- A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithm…☆68Updated last month
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆73Updated last year
- ☆22Updated 5 months ago
- Quantized Attention on GPU☆44Updated 10 months ago
- Fast and memory-efficient exact kmeans☆100Updated last week
- Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"☆120Updated last year
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆119Updated 3 months ago
- ☆30Updated last month
- Patch convolution to avoid large GPU memory usage of Conv2D☆92Updated 8 months ago
- Efficient triton implementation of Native Sparse Attention.☆229Updated 4 months ago
- A collection of tricks and tools to speed up transformer models☆182Updated this week
- [ICLR‘24 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"☆95Updated 3 months ago