Alternatives and similar repositories for Ray-Connect

Users that are interested in Ray-Connect are comparing it to the libraries listed below

• antgroup / ant-ray
  Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. AntRay i…
  ☆165Updated this week
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆131Updated 4 months ago
• alibaba / TePDist
  TePDist (TEnsor Program DISTributed) is an HLO-level automatic distributed system for DL models.
  ☆99Updated 2 years ago
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆60Updated last year
• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆263Updated this week
• yale-sys / prompt-cache
  Modular and structured prompt caching for low-latency LLM inference
  ☆110Updated last year
• ray-project / mobius
  Mobius is an AI infrastructure platform for distributed online learning, including online sample processing, training and serving.
  ☆100Updated last year
• antgroup / glake
  GLake: optimizing GPU memory management and IO transmission.
  ☆497Updated 10 months ago
• ray-project / Ray-Forward
  Some resources about Ray Forward Meetup
  ☆30Updated last month
• bentoml / llm-bench
  ☆56Updated last year
• AlibabaPAI / torchacc
  PyTorch distributed training acceleration framework
  ☆55Updated 5 months ago
• DeepRec-AI / serving
  A high-performance serving system for DeepRec based on TensorFlow Serving.
  ☆19Updated 2 years ago
• Tencent / KsanaLLM
  ☆523Updated 2 weeks ago
• OpenSQZ / MegatronApp
  Toolchain built around the Megatron-LM for Distributed Training
  ☆84Updated 2 months ago
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆524Updated 5 months ago
• 4paradigm / OpenEmbedding
  OpenEmbedding is an open source framework for Tensorflow distributed training acceleration.
  ☆33Updated 2 years ago
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆120Updated last year
• eddiegaoo / Apt-Serve
  ☆20Updated 8 months ago
• alibaba / EasyParallelLibrary
  Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
  ☆271Updated 2 years ago
• LMCache / demo
  ☆27Updated 9 months ago
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆390Updated 2 months ago
• WukLab / preble
  Stateful LLM Serving
  ☆95Updated 10 months ago
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆83Updated 11 months ago
• run-ai / llmperf
  ☆61Updated last year
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆470Updated last month
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆135Updated last year
• inclusionAI / asystem-awex
  A high-performance RL training-inference weight synchronization framework, designed to enable second-level parameter updates from trainin…
  ☆131Updated last month
• tyler-griggs / melange-release
  ☆48Updated last year
• ovg-project / kvcached
  Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
  ☆773Updated this week
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆209Updated last year