ruipeterpan / marconiLinks
Artifact for "Marconi: Prefix Caching for the Era of Hybrid LLMs" [MLSys '25 Outstanding Paper Honorable Mention]
☆11Updated 3 months ago
Alternatives and similar repositories for marconi
Users that are interested in marconi are comparing it to the libraries listed below
Sorting:
- NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading☆39Updated last week
- LLM serving cluster simulator☆106Updated last year
- Proteus: A High-Throughput Inference-Serving System with Accuracy Scaling☆13Updated last year
- Open-source implementation for "Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow"☆49Updated 7 months ago
- ☆62Updated last year
- ☆21Updated last year
- LLM Inference analyzer for different hardware platforms