mdy666 / mdy_tritonLinks
☆149Updated 5 months ago
Alternatives and similar repositories for mdy_triton
Users that are interested in mdy_triton are comparing it to the libraries listed below
Sorting:
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length☆137Updated last month
- ☆154Updated 9 months ago
- Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)☆344Updated 8 months ago
- qwen-nsa☆85Updated 2 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆360Updated 5 months ago
- Implement some method of LLM KV Cache Sparsity☆41Updated last year
- Implementation of FlashAttention in PyTorch☆176Updated 11 months ago
- ☆45Updated last year
- ☆111Updated 3 months ago
- A collection of memory efficient attention operators implemented in the Triton language.☆287Updated last year
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.☆557Updated last week
- ☆296Updated 5 months ago
- [NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.☆490Updated last year
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆300Updated 6 months ago
- [ICML 2025] Official PyTorch implementation of "FlatQuant: Flatness Matters for LLM Quantization"☆200Updated last month
- Puzzles for learning Triton, play it with minimal environment configuration!☆576Updated 3 weeks ago
- ☆58Updated last year
- Efficient Mixture of Experts for LLM Paper List☆149Updated 2 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆174Updated last week
- An easy-to-use package for implementing SmoothQuant for LLMs☆110Updated 8 months ago
- The Official Implementation of Ada-KV [NeurIPS 2025]☆120Updated 3 weeks ago
- Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**☆215Updated 10 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆180Updated 3 months ago
- [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache☆343Updated last month
- Implement Flash Attention using Cute.☆98Updated last year
- [ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projection☆149Updated 10 months ago
- [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.☆132Updated last year
- Awesome list for LLM quantization☆371Updated 2 months ago
- Awesome list for LLM pruning.☆279Updated 2 months ago
- llm theoretical performance analysis tools and support params, flops, memory and latency analysis.☆113Updated 5 months ago