Alternatives and similar repositories for Ray-Connect

Users that are interested in Ray-Connect are comparing it to the libraries listed below

• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆131Updated last month
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆58Updated 9 months ago
• antgroup / ant-ray
  Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. AntRay i…
  ☆157Updated this week
• yale-sys / prompt-cache
  Modular and structured prompt caching for low-latency LLM inference
  ☆102Updated last year
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆80Updated 8 months ago
• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆230Updated this week
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆119Updated last year
• InternLM / turbomind
  ☆97Updated 7 months ago
• bentoml / llm-bench
  ☆56Updated last year
• run-ai / llmperf
  ☆57Updated last year
• ninehills / llm-inference-benchmark
  LLM Inference benchmark
  ☆429Updated last year
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆510Updated 2 months ago
• omni-ai-npu / omni-infer
  Omni_Infer is a suite of inference accelerators designed for the Ascend NPU platform, offering native support and an expanding feature se…
  ☆86Updated this week
• alibaba / TePDist
  TePDist (TEnsor Program DISTributed) is an HLO-level automatic distributed system for DL models.
  ☆97Updated 2 years ago
• alibaba / EasyParallelLibrary
  Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
  ☆270Updated 2 years ago
• OpenPPL / ppl.llm.serving
  ☆130Updated 10 months ago
• tyler-griggs / melange-release
  ☆48Updated last year
• simon-mo / vLLM-Benchmark
  ☆31Updated 7 months ago
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆445Updated last month
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆268Updated 3 months ago
• AlibabaPAI / torchacc
  PyTorch distributed training acceleration framework
  ☆53Updated 3 months ago
• LMCache / demo
  ☆27Updated 7 months ago
• antgroup / glake
  GLake: optimizing GPU memory management and IO transmission.
  ☆489Updated 7 months ago
• WukLab / preble
  Stateful LLM Serving
  ☆88Updated 8 months ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆79Updated last year
• intel / xFasterTransformer
  ☆431Updated 2 months ago
• ray-project / mobius
  Mobius is an AI infrastructure platform for distributed online learning, including online sample processing, training and serving.
  ☆100Updated last year
• vllm-project / recipes
  Common recipes to run vLLM
  ☆236Updated last week
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆361Updated last week
• Tencent / KsanaLLM
  ☆512Updated 2 months ago