ml-energy / leaderboard-v2Links
A canonical source of GenAI energy benchmark and meausrements
☆50Updated this week
Alternatives and similar repositories for leaderboard-v2
Users that are interested in leaderboard-v2 are comparing it to the libraries listed below
Sorting:
- Measure and optimize the energy consumption of your AI applications!☆311Updated last week
- A resilient distributed training framework☆96Updated last year
- PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".☆94Updated 2 years ago
- LLM Serving Performance Evaluation Harness☆81Updated 9 months ago
- ☆25Updated 2 years ago
- ☆48Updated last year
- End-to-end carbon footprint mod- eling tool☆51Updated 6 months ago
- Artifact for "Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving" [SOSP '24]☆25Updated last year
- ☆94Updated 3 years ago
- Easy, Fast, and Scalable Multimodal AI☆73Updated this week
- ACT An Architectural Carbon Modeling Tool for Designing Sustainable Computer Systems☆43Updated 4 months ago
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆68Updated 11 months ago
- Official Repo for "SplitQuant / LLM-PQ: Resource-Efficient LLM Offline Serving on Heterogeneous GPUs via Phase-Aware Model Partition and …☆35Updated 3 months ago
- Surrogate-based Hyperparameter Tuning System☆27Updated 2 years ago
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆132Updated last year
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆199Updated last year
- ☆79Updated last month
- [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration☆243Updated last year
- ☆20Updated 2 years ago
- Stateful LLM Serving☆89Updated 8 months ago
- LLM checkpointing for DeepSpeed/Megatron☆21Updated last month
- Code for MLSys 2024 Paper "SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models"☆21Updated last year
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆43Updated 3 years ago
- ☆45Updated 7 months ago
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank☆64Updated last year
- Compression for Foundation Models☆34Updated 4 months ago
- ☆88Updated 3 years ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆132Updated 11 months ago
- [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention☆49Updated 3 months ago
- A framework for generating realistic LLM serving workloads☆82Updated last month