mdy666 / mdy_tritonLinks
☆131Updated last month
Alternatives and similar repositories for mdy_triton
Users that are interested in mdy_triton are comparing it to the libraries listed below
Sorting:
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length☆83Updated last month
- ☆137Updated 2 months ago
- The Official Implementation of Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference☆76Updated 4 months ago
- Implement some method of LLM KV Cache Sparsity☆32Updated 11 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆102Updated this week
- Implement Flash Attention using Cute.☆85Updated 5 months ago
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆107Updated 2 weeks ago
- A sparse attention kernel supporting mix sparse patterns☆219Updated 3 months ago
- ☆93Updated 2 weeks ago
- Puzzles for learning Triton, play it with minimal environment configuration!☆334Updated 6 months ago
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training☆203Updated 2 weeks ago
- qwen-nsa☆66Updated last month
- [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.☆114Updated last year
- Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)☆269Updated last month
- [NeurIPS 2024 Oral🔥] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs.☆161Updated 7 months ago
- Multi-Candidate Speculative Decoding☆35Updated last year
- ☆36Updated 9 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆291Updated 6 months ago
- 📚FFPA(Split-D): Extend FlashAttention with Split-D for large headdim, O(1) GPU SRAM complexity, 1.8x~3x↑🎉 faster than SDPA EA.☆183Updated 3 weeks ago
- ☆248Updated last year
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆198Updated last week
- Efficient Mixture of Experts for LLM Paper List☆68Updated 5 months ago
- flash attention tutorial written in python, triton, cuda, cutlass☆368Updated 2 weeks ago
- 16-fold memory access reduction with nearly no loss☆94Updated 2 months ago
- Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**☆186Updated 3 months ago
- Implementation of FlashAttention in PyTorch☆150Updated 4 months ago
- Awesome list for LLM quantization☆223Updated 5 months ago
- QAQ: Quality Adaptive Quantization for LLM KV Cache☆50Updated last year
- This repository serves as a comprehensive survey of LLM development, featuring numerous research papers along with their corresponding co…☆144Updated 3 months ago
- 🔥 A minimal training framework for scaling FLA models☆146Updated 2 weeks ago