infinigence / SpecEELinks
Repo for SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting (ISCA25)
☆37Updated 2 months ago
Alternatives and similar repositories for SpecEE
Users that are interested in SpecEE are comparing it to the libraries listed below
Sorting:
- A lightweight design for computation-communication overlap.☆143Updated this week
- ☆77Updated last month
- Examples of CUDA implementations by Cutlass CuTe☆197Updated 4 months ago
- [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"☆54Updated 2 weeks ago
- High performance Transformer implementation in C++.☆125Updated 5 months ago
- A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆47Updated 2 weeks ago
- [ICML 2025] Official PyTorch implementation of "FlatQuant: Flatness Matters for LLM Quantization"☆138Updated last month
- Summary of some awesome work for optimizing LLM inference☆77Updated 3 weeks ago
- ☆148Updated 5 months ago
- LLM Inference analyzer for different hardware platforms☆74Updated 3 weeks ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆87Updated last month
- Optimize softmax in triton in many cases☆21Updated 9 months ago
- ☆69Updated this week
- ☆123Updated 6 months ago
- llm theoretical performance analysis tools and support params, flops, memory and latency analysis.☆96Updated last week
- Code Repository of Evaluating Quantized Large Language Models☆124Updated 9 months ago
- QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.☆128Updated 2 months ago
- ☆147Updated 11 months ago
- ☆96Updated 9 months ago
- Implement Flash Attention using Cute.☆87Updated 6 months ago
- ☆110Updated 3 weeks ago
- ☆58Updated 7 months ago
- ☆146Updated 6 months ago
- ☆103Updated 7 months ago
- [USENIX ATC '24] Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Paral…☆55Updated 10 months ago
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆255Updated 3 months ago
- Curated collection of papers in MoE model inference☆197Updated 4 months ago
- ⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.☆80Updated last month
- DeepSeek-V3/R1 inference performance simulator☆149Updated 2 months ago
- ☆39Updated last year