infinigence / SpecEELinks
Repo for SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting (ISCA25)
☆45Updated 2 months ago
Alternatives and similar repositories for SpecEE
Users that are interested in SpecEE are comparing it to the libraries listed below
Sorting:
- High performance Transformer implementation in C++.☆125Updated 6 months ago
- Summary of some awesome work for optimizing LLM inference☆84Updated last month
- ☆23Updated last year
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)☆142Updated last year
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆259Updated 4 months ago
- ☆113Updated 2 weeks ago
- ☆149Updated 11 months ago
- Curated collection of papers in MoE model inference☆210Updated 4 months ago
- Explore Inter-layer Expert Affinity in MoE Model Inference☆11Updated last year
- ☆19Updated 3 months ago
- ☆41Updated last year
- ☆106Updated 8 months ago
- A lightweight design for computation-communication overlap.☆146Updated 3 weeks ago
- LLM Inference analyzer for different hardware platforms☆79Updated last week
- A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆52Updated last week
- llm theoretical performance analysis tools and support params, flops, memory and latency analysis.☆98Updated this week
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆234Updated last month
- This repository is established to store personal notes and annotated papers during daily research.☆135Updated this week
- ☆64Updated last year
- ☆19Updated 3 months ago
- ☆168Updated last year
- DeepSeek-V3/R1 inference performance simulator☆155Updated 3 months ago
- Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline mod…☆509Updated 10 months ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆95Updated 2 months ago
- This repository serves as a comprehensive survey of LLM development, featuring numerous research papers along with their corresponding co…☆164Updated 3 weeks ago
- Optimize softmax in triton in many cases☆21Updated 10 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆305Updated last week
- LLM serving cluster simulator☆107Updated last year
- ☆55Updated last year
- [ICML 2025] Official PyTorch implementation of "FlatQuant: Flatness Matters for LLM Quantization"☆141Updated last month