Alternatives and similar repositories for prism-research

Users that are interested in prism-research are comparing it to the libraries listed below

• WukLab / preble
  Stateful LLM Serving
  ☆81Updated 5 months ago
• hao-ai-lab / vllm-ltr
  [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank
  ☆58Updated 10 months ago
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆181Updated 11 months ago
• YaoJiayi / CacheBlend
  ☆131Updated last month
• LLMServe / SwiftTransformer
  High performance Transformer implementation in C++.
  ☆129Updated 7 months ago
• hao-ai-lab / MuxServe
  ☆69Updated last year
• ovg-project / kvcached
  kvcached: Elastic KV cache for dynamic GPU sharing and efficient multi-LLM inference.
  ☆66Updated this week
• alibaba / ServeGen
  A framework for generating realistic LLM serving workloads
  ☆54Updated 2 months ago
• microsoft / chunk-attention
  ☆78Updated 4 months ago
• LoongServe / LoongServe
  ☆116Updated 9 months ago
• ruipeterpan / marconi
  Artifact for "Marconi: Prefix Caching for the Era of Hybrid LLMs" [MLSys '25 Outstanding Paper Award, Honorable Mention]
  ☆21Updated 5 months ago
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆128Updated last year
• stepfun-ai / StepMesh
  ☆288Updated last week
• UChi-JCL / CacheGen
  ☆123Updated 10 months ago
• Hsword / Awesome-Machine-Learning-System-Papers
  ☆75Updated 3 years ago
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆112Updated last year
• microsoft / RetrievalAttention
  Scalable long-context LLM decoding that leverages sparsity—by treating the KV cache as a vector storage system.
  ☆77Updated 2 weeks ago
• interestingLSY / swiftLLM
  A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …
  ☆249Updated 2 months ago
• yichuan-w / MLsys_reading_list
  A record of reading list on some MLsys popular topic
  ☆15Updated 5 months ago
• infinigence / Semi-PD
  A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
  ☆105Updated 3 months ago
• flexflow / flexflow-serve
  FlexFlow Serve: Low-Latency, High-Performance LLM Serving
  ☆56Updated this week
• gty111 / gLLM
  gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
  ☆39Updated last week
• fzyzcjy / torch_memory_saver
  Allow torch tensor memory to be released and resumed later
  ☆118Updated this week
• EfficientMoE / MoE-Infinity
  PyTorch library for cost-effective, fast and easy serving of MoE models.
  ☆228Updated last month
• eddiegaoo / Apt-Serve
  ☆16Updated 2 months ago
• infinigence / FlashOverlap
  A lightweight design for computation-communication overlap.
  ☆160Updated last week
• InternLM / Awesome-LLM-Training-System
  ☆42Updated last year
• galeselee / Awesome_LLM_System-PaperList
  Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…
  ☆266Updated 5 months ago
• NEO-MLSys25 / NEO
  NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading
  ☆56Updated 2 months ago
• fanlai0990 / CS598
  Systems for GenAI
  ☆144Updated 4 months ago