PKU-SEC-Lab / HybriMoE
[DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"
☆18Updated last week
Alternatives and similar repositories for HybriMoE
Users that are interested in HybriMoE are comparing it to the libraries listed below
Sorting:
- ☆10Updated 3 months ago
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)☆130Updated 10 months ago
- ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction (NIPS'24)☆38Updated 4 months ago
- ☆54Updated last year
- [ICML 2024] Serving LLMs on heterogeneous decentralized clusters.☆25Updated last year
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆44Updated last month
- High performance Transformer implementation in C++.☆122Updated 3 months ago
- [SIGMOD 2025] PQCache: Product Quantization-based KVCache for Long Context LLM Inference☆46Updated 3 months ago
- Code release for AdapMoE accepted by ICCAD 2024☆23Updated 2 weeks ago
- ☆96Updated 6 months ago
- This repository serves as a comprehensive survey of LLM development, featuring numerous research papers along with their corresponding co…☆125Updated 2 months ago
- ☆60Updated 11 months ago
- ☆111Updated 4 months ago
- Summary of some awesome work for optimizing LLM inference☆73Updated last month
- This repository is established to store personal notes and annotated papers during daily research.☆122Updated 3 weeks ago
- ☆13Updated 10 months ago
- Curated collection of papers in MoE model inference☆174Updated 2 months ago
- A ChatGPT(GPT-3.5) & GPT-4 Workload Trace to Optimize LLM Serving Systems☆167Updated 7 months ago
- LLM Inference analyzer for different hardware platforms☆66Updated 2 weeks ago
- ☆73Updated 3 years ago
- ☆17Updated last year
- ☆102Updated 7 months ago
- LLM serving cluster simulator☆99Updated last year
- The official implementation for the intra-stage fusion technique introduced in https://arxiv.org/abs/2409.13221☆14Updated 3 weeks ago
- A lightweight design for computation-communication overlap.☆113Updated last week
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank☆48Updated 6 months ago
- Implement Flash Attention using Cute.☆82Updated 4 months ago
- Artifact of OSDI '24 paper, ”Llumnix: Dynamic Scheduling for Large Language Model Serving“☆60Updated 11 months ago
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆250Updated 2 months ago
- Stateful LLM Serving☆67Updated 2 months ago