Alternatives and similar repositories for Ray-Connect

Users that are interested in Ray-Connect are comparing it to the libraries listed below

• antgroup / ant-ray
  Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. AntRay i…
  ☆162Updated this week
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆59Updated 11 months ago
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆131Updated 3 months ago
• ray-project / mobius
  Mobius is an AI infrastructure platform for distributed online learning, including online sample processing, training and serving.
  ☆99Updated last year
• 4paradigm / OpenEmbedding
  OpenEmbedding is an open source framework for Tensorflow distributed training acceleration.
  ☆33Updated 2 years ago
• ray-project / Ray-Forward
  Some resources about Ray Forward Meetup
  ☆28Updated 2 weeks ago
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆82Updated 10 months ago
• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆251Updated this week
• alibaba / TePDist
  TePDist (TEnsor Program DISTributed) is an HLO-level automatic distributed system for DL models.
  ☆99Updated 2 years ago
• bentoml / llm-bench
  ☆56Updated last year
• yale-sys / prompt-cache
  Modular and structured prompt caching for low-latency LLM inference
  ☆109Updated last year
• LMCache / demo
  ☆27Updated 8 months ago
• AlibabaPAI / torchacc
  PyTorch distributed training acceleration framework
  ☆54Updated 4 months ago
• run-ai / llmperf
  ☆60Updated last year
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆120Updated last year
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆520Updated 4 months ago
• InternLM / turbomind
  ☆96Updated 9 months ago
• tensorchord / modelz-ChatGLM
  Deploy ChatGLM on Modelz
  ☆16Updated 2 years ago
• WukLab / preble
  Stateful LLM Serving
  ☆93Updated 10 months ago
• OpenSQZ / MegatronApp
  Toolchain built around the Megatron-LM for Distributed Training
  ☆80Updated last month
• inclusionAI / asystem-awex
  A high-performance RL training-inference weight synchronization framework, designed to enable second-level parameter updates from trainin…
  ☆123Updated 2 weeks ago
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆384Updated last month
• omni-ai-npu / omni-infer
  Omni_Infer is a suite of inference accelerators designed for the Ascend NPU platform, offering native support and an expanding feature se…
  ☆96Updated this week
• alibaba / EasyParallelLibrary
  Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
  ☆271Updated 2 years ago
• anyscale / llm-continuous-batching-benchmarks
  ☆123Updated last year
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆134Updated last year
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆363Updated last week
• bytedance / primus
  ☆222Updated 2 years ago
• antgroup / glake
  GLake: optimizing GPU memory management and IO transmission.
  ☆494Updated 9 months ago
• zijie0 / cloud_ml_platform
  ☆35Updated 4 years ago