kyegomez / FlashAttention20Triton
Triton implementation of Flash Attention2.0
☆26Updated last year
Alternatives and similar repositories for FlashAttention20Triton:
Users that are interested in FlashAttention20Triton are comparing it to the libraries listed below
- Odysseus: Playground of LLM Sequence Parallelism☆64Updated 7 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆58Updated 2 months ago
- ☆72Updated 5 months ago
- The official code for paper "parallel speculative decoding with adaptive draft length."☆32Updated 4 months ago
- Summary of system papers/frameworks/codes/tools on training or serving large model☆56Updated last year
- QAQ: Quality Adaptive Quantization for LLM KV Cache☆44Updated 9 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆84Updated 2 weeks ago
- [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…☆47Updated 6 months ago
- 16-fold memory access reduction with nearly no loss☆63Updated 2 months ago
- Official PyTorch implementation of FlatQuant: Flatness Matters for LLM Quantization☆89Updated 2 months ago
- Vocabulary Parallelism☆16Updated 2 months ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆152Updated 6 months ago
- An easy-to-use package for implementing SmoothQuant for LLMs☆89Updated 8 months ago
- Multi-Candidate Speculative Decoding☆33Updated 8 months ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆64Updated 4 months ago
- An algorithm for static activation quantization of LLMs☆107Updated last week
- ☆54Updated last month
- QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.☆94Updated last month
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)☆85Updated 3 months ago
- ☆96Updated 4 months ago
- The Official Implementation of Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference☆54Updated last month
- Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs☆76Updated last month
- Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆107Updated last month
- ☆56Updated 3 months ago
- Official implementation of the ICLR 2024 paper AffineQuant☆23Updated 9 months ago
- ☆59Updated last month
- Sequence-level 1F1B schedule for LLMs.☆17Updated 7 months ago
- Quantized Attention on GPU☆34Updated last month
- AFPQ code implementation☆19Updated last year
- ☆51Updated 9 months ago