Predict the performance of LLM inference services
☆23Sep 18, 2025Updated last year
Alternatives and similar repositories for LLM-performance-prediction
Users that are interested in LLM-performance-prediction are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆140Feb 22, 2024Updated 2 years ago
- Cloud Native Benchmarking of Foundation Models☆46Jul 31, 2025Updated last year
- AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving (OSDI 23)☆94Jul 14, 2023Updated 3 years ago
- Artifact for "Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving" [SOSP '24]☆25Nov 21, 2024Updated last year
- Serverless Paper Reading and Discussion☆37Jan 9, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- latency optimization with priority optimization☆12Jun 18, 2025Updated last year
- ☆13Nov 5, 2025Updated last year
- LangBench applications and scripts☆14Jun 7, 2023Updated 3 years ago
- ☆186Mar 12, 2024Updated 2 years ago
- ☆10Jul 23, 2024Updated 2 years ago
- Failure dataset accompanying the paper "How Bad Can a Bug Get? An Empirical Analysis of Software Failures in the OpenStack Cloud Computi…☆11Jun 12, 2020Updated 6 years ago
- Simulator for the datacenter, including power, cooling, server and other components☆19Feb 12, 2025Updated last year
- Official Repo for "SplitQuant / LLM-PQ: Resource-Efficient LLM Offline Serving on Heterogeneous GPUs via Phase-Aware Model Partition and …☆41Aug 29, 2025Updated last year
- Tool to calculate CCF coefficients based on SPECpower results.☆12Apr 24, 2026Updated 6 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Source code for OSDI 2023 paper titled "Cilantro - Performance-Aware Resource Allocation for General Objectives via Online Feedback"☆40Jul 6, 2023Updated 3 years ago
- Repo for OpenCarbonEval: A Unified Carbon Emission Estimation Framework in Large-Scale AI Models☆11May 21, 2024Updated 2 years ago
- Accurate, large-scale, and extensible simulator for LLM inference Systems☆732Aug 24, 2026Updated 2 months ago
- A tool to detect infrastructure issues on cloud native AI systems☆56Sep 18, 2025Updated last year
- ☆17Apr 7, 2025Updated last year
- ☆136Nov 11, 2024Updated 2 years ago
- ☆13Jun 20, 2025Updated last year
- ☆49Jun 27, 2024Updated 2 years ago
- A ChatGPT(GPT-3.5) & GPT-4 Workload Trace to Optimize LLM Serving Systems☆298Jun 30, 2026Updated 4 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆16Jan 14, 2025Updated last year
- ☆18Oct 31, 2022Updated 4 years ago
- Prism: Revealing Hidden Functional Clusters from Massive Instances in Cloud Systems [ASE'2023]☆10Jan 10, 2024Updated 2 years ago
- LLM Serving Performance Evaluation Harness☆87Feb 25, 2025Updated last year
- ☆28Nov 8, 2022Updated 4 years ago
- TraceWeaver is a research prototype for transparently tracing requests through a microservice without application instrumentation.☆23Sep 2, 2024Updated 2 years ago
- Releasing the spot availability traces used in "Can't Be Late" paper.☆27Mar 31, 2024Updated 2 years ago
- Know Your Enemy To Save Cloud Energy: Energy-Performance Characterization of Machine Learning Serving (HPCA '23)☆14Jun 20, 2025Updated last year
- ☆32May 28, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆47Jul 4, 2024Updated 2 years ago
- ☆20Sep 25, 2023Updated 3 years ago
- ☆10Dec 10, 2024Updated last year
- JellyBean prototype☆10Sep 28, 2022Updated 4 years ago
- A library developed by Volcano Engine for high-performance reading and writing of PyTorch model files.☆27Jan 2, 2025Updated last year
- Source Code for ISSRE-24 paper "Demystifying and Extracting Fault-indicating Information from Logs for Failure Diagnosis".☆12Jan 4, 2025Updated last year
- ☆54Oct 4, 2025Updated last year
Are these results useful?