thu-pacman / SmartMoE-AE
ATC23 AE
☆45Updated last year
Alternatives and similar repositories for SmartMoE-AE:
Users that are interested in SmartMoE-AE are comparing it to the libraries listed below
- ☆82Updated 3 years ago
- nnScaler: Compiling DNN models for Parallel Training☆107Updated last week
- ☆95Updated 5 months ago
- [USENIX ATC '24] Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Paral…☆52Updated 8 months ago
- PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".☆85Updated last year
- ☆72Updated 3 years ago
- ☆69Updated last week
- ☆53Updated last year
- Summary of system papers/frameworks/codes/tools on training or serving large model☆56Updated last year
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆97Updated last year
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length☆77Updated last week
- Implement some method of LLM KV Cache Sparsity☆31Updated 10 months ago
- [ICML 2024] Serving LLMs on heterogeneous decentralized clusters.☆25Updated 11 months ago
- A high-performance distributed deep learning system targeting large-scale and automated distributed training. If you have any interests, …☆110Updated last year
- ☆131Updated last month
- ☆92Updated 7 months ago
- ☆140Updated 9 months ago
- The Official Implementation of Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference☆72Updated 3 months ago
- ☆59Updated 10 months ago
- 16-fold memory access reduction with nearly no loss☆90Updated 3 weeks ago
- A simple calculation for LLM MFU.☆36Updated last month
- Explore Inter-layer Expert Affinity in MoE Model Inference☆8Updated 11 months ago
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆154Updated 7 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆68Updated 10 months ago
- Official repository for the paper DynaPipe: Optimizing Multi-task Training through Dynamic Pipelines☆17Updated last year
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆159Updated 9 months ago
- High performance Transformer implementation in C++.☆118Updated 3 months ago
- A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆33Updated 3 weeks ago
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank☆46Updated 5 months ago
- Automated Parallelization System and Infrastructure for Multiple Ecosystems☆78Updated 5 months ago