merrymercy / Awesome-Efficient-LLMLinks
A curated list for Efficient Large Language Models
☆11Updated last year
Alternatives and similar repositories for Awesome-Efficient-LLM
Users that are interested in Awesome-Efficient-LLM are comparing it to the libraries listed below
Sorting:
- Artifact for "Marconi: Prefix Caching for the Era of Hybrid LLMs" [MLSys '25 Outstanding Paper Award, Honorable Mention]☆32Updated 6 months ago
- ☆19Updated 11 months ago
- FractalTensor is a programming framework that introduces a novel approach to organizing data in deep neural networks (DNNs) as a list of …☆27Updated 9 months ago
- [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention☆47Updated last month
- PyTorch compilation tutorial covering TorchScript, torch.fx, and Slapo☆17Updated 2 years ago
- Framework to reduce autotune overhead to zero for well known deployments.☆82Updated last week
- ☆21Updated 3 years ago
- ☆33Updated last year
- An Attention Superoptimizer☆22Updated 8 months ago
- Efficient Compute-Communication Overlap for Distributed LLM Inference☆43Updated 2 weeks ago
- ☆20Updated 2 years ago
- LLM-Inference-Bench☆52Updated 2 months ago
- ☆15Updated last year
- ☆82Updated 8 months ago
- ☆38Updated last year
- GPTQ inference TVM kernel☆40Updated last year
- An external memory allocator example for PyTorch.☆16Updated last month
- ☆71Updated last year
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆57Updated 6 months ago
- Compression for Foundation Models☆35Updated 2 months ago
- DeeperGEMM: crazy optimized version☆70Updated 4 months ago