infinigence / SpecEELinks
Repo for SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting (ISCA25)
☆70Updated 9 months ago
Alternatives and similar repositories for SpecEE
Users that are interested in SpecEE are comparing it to the libraries listed below
Sorting:
- [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"☆101Updated last month
- [HPCA 2026] A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆80Updated last month
- Summary of some awesome work for optimizing LLM inference☆173Updated 2 months ago
- ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction (NIPS'24)☆53Updated last year
- ☆26Updated last year
- High performance Transformer implementation in C++.☆150Updated last year
- LLM Inference analyzer for different hardware platforms☆100Updated 2 months ago
- ☆145Updated last month
- A lightweight design for computation-communication overlap.☆219Updated 3 weeks ago
- Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding☆87Updated 2 months ago
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)☆174Updated last year
- Curated collection of papers in MoE model inference☆341Updated 3 months ago
- ☆131Updated last year
- [NeurIPS 2025] ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive☆66Updated 2 months ago
- ☆58Updated last year
- ☆164Updated last year
- Explore Inter-layer Expert Affinity in MoE Model Inference☆16Updated last year
- ☆95Updated 10 months ago
- Code Repository of Evaluating Quantized Large Language Models☆136Updated last year
- ☆85Updated 3 months ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆123Updated last month
- ☆224Updated 3 months ago
- ☆31Updated 10 months ago
- Code release for AdapMoE accepted by ICCAD 2024☆35Updated 9 months ago
- LLM serving cluster simulator☆135Updated last year
- DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit☆92Updated 2 weeks ago
- Tile-based language built for AI computation across all scales☆120Updated this week
- DeepSeek-V3/R1 inference performance simulator☆176Updated 10 months ago
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆283Updated 11 months ago
- Learning TileLang with 10 puzzles!☆118Updated last week