A framework for generating realistic LLM serving workloads
☆188Aug 27, 2026Updated 2 months ago
Alternatives and similar repositories for ServeGen
Users that are interested in ServeGen are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆184Apr 23, 2026Updated 6 months ago
- A ChatGPT(GPT-3.5) & GPT-4 Workload Trace to Optimize LLM Serving Systems☆292Jun 30, 2026Updated 4 months ago
- Repo to replay Qwen trace☆36Jan 9, 2026Updated 9 months ago
- Research prototype of PRISM — a cost-efficient multi-LLM serving system with flexible time- and space-based GPU sharing.☆77Mar 17, 2026Updated 7 months ago
- Accurate, large-scale, and extensible simulator for LLM inference Systems☆717Aug 24, 2026Updated 2 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Open-source implementation for "Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow"☆98Oct 15, 2025Updated last year
- Efficient and easy multi-instance LLM serving☆561Mar 12, 2026Updated 7 months ago
- A low-latency & high-throughput serving engine for LLMs☆535Jan 8, 2026Updated 9 months ago
- LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure☆448Oct 22, 2026Updated last week
- ☆186Mar 12, 2024Updated 2 years ago
- ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale☆715Updated this week
- ☆137Nov 11, 2024Updated last year
- Large Language Model (LLM) Systems Paper List☆2,309Oct 20, 2026Updated last week
- This is the code for “SMOG: Accelerating Subgraph Matching on GPUs”, which won the GraphChallenge 2023 champion.☆11Oct 16, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆23Jul 13, 2026Updated 3 months ago
- Easy, Fast, and Scalable Multimodal AI☆135Jun 2, 2026Updated 4 months ago
- Disaggregated serving system for Large Language Models (LLMs).☆845Apr 6, 2025Updated last year
- ☆93Oct 17, 2025Updated last year
- Curated collection of papers in machine learning systems☆657Oct 17, 2026Updated 2 weeks ago
- DeDe (OSDI '25): an optimization framework for large-scale resource allocation☆15May 18, 2026Updated 5 months ago
- Medusa: Accelerating Serverless LLM Inference with Materialization [ASPLOS'25]☆50May 13, 2025Updated last year
- ☆24Sep 10, 2025Updated last year
- ☆11Updated this week
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- [ASPLOS' 26] TetriServe: Efficiently Serving Mixed DiT Workloads☆20Mar 12, 2026Updated 7 months ago
- ☆1,222Sep 20, 2026Updated last month
- Code repository for the SOSP'25 paper DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism.☆23Nov 28, 2025Updated 11 months ago
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆363Jun 10, 2025Updated last year
- Resource Multiplexing in Tuning and Serving Large Language Models (USENIX ATC 2025)☆10Apr 21, 2026Updated 6 months ago
- This is the implementation repository of our SOSP'24 paper: Aceso: Achieving Efficient Fault Tolerance in Memory-Disaggregated Key-Value …☆24Oct 20, 2024Updated 2 years ago
- Mini-SGLang is a mini version of SGLang project ().☆13Sep 24, 2025Updated last year
- A programmable system for high-performance model serving☆238Updated this week
- A lightweight CuTe-based CUDA kernel for DuoAttention, optimized for large language model inference.☆12Dec 25, 2025Updated 10 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- GPU-accelerated LLM Training Simulator☆55Jun 26, 2025Updated last year
- A simple, fast and robust program-aware agentic inference system.☆471Jul 5, 2026Updated 3 months ago
- ☆67Jun 25, 2024Updated 2 years ago
- Dynamic Memory Management for Serving LLMs without PagedAttention☆526Aug 24, 2026Updated 2 months ago
- Opal (O.P.A.L. - Open simulator Platform for distributed AI and LLM workflows) is an LLM platform-level simulator written purely in Pytho…☆18Oct 22, 2026Updated last week
- Microsoft Azure Traces☆1,220Oct 1, 2026Updated 3 weeks ago
- ☆17Jun 14, 2026Updated 4 months ago