Pretrain, finetune and serve LLMs on Intel platforms with Ray
☆129Sep 23, 2025Updated last year
Alternatives and similar repositories for llm-on-ray
Users that are interested in llm-on-ray are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- RayLLM - LLMs on Ray (Archived). Read README for more info.☆1,260Mar 13, 2025Updated last year
- ☆15Mar 3, 2025Updated last year
- DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.☆14Jan 8, 2026Updated 9 months ago
- Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. AntRay i…☆174Aug 8, 2026Updated 2 months ago
- A modular acceleration toolkit for big data analytic engines☆66May 6, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Implementation of algorithms for memory optimized deep neural network training☆11Jul 23, 2020Updated 6 years ago
- Material for Ray Connect 2024 Conference☆12Oct 23, 2024Updated 2 years ago
- Intel® End-to-End AI Optimization Kit☆30Jul 18, 2024Updated 2 years ago
- Document Automation Reference Kit☆16Jun 27, 2024Updated 2 years ago
- A std::execution style runtime context and High Performance RPC Transport for using OpenUCX. Including CUDA/ROCM/... devices with RDMA.☆34Oct 14, 2026Updated 3 weeks ago
- LLM query engine to retrieve augmented responses from json files.☆15Oct 12, 2023Updated 3 years ago
- MLFlow Deployment Plugin for Ray Serve☆47Apr 12, 2022Updated 4 years ago
- OpenAI Triton backend for Intel® GPUs☆275Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆92Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Analysis for the traces from byteprofile☆32Nov 21, 2023Updated 2 years ago
- GLake: optimizing GPU memory management and IO transmission.☆502Mar 24, 2025Updated last year
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆140Feb 22, 2024Updated 2 years ago
- ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…☆2,174Oct 8, 2024Updated 2 years ago
- ☆128Dec 24, 2024Updated last year
- oneCCL Bindings for Pytorch* (deprecated)☆104Dec 31, 2025Updated 10 months ago
- A toolkit to run Ray applications on Kubernetes☆2,759Updated this week
- ☆49Jun 27, 2024Updated 2 years ago
- Tengine 管子是用来快速生产 demo 的辅助工具☆11Jul 15, 2021Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Another LaTex formula OCR tool☆15Feb 15, 2023Updated 3 years ago
- A multiple parties joint, distributed execution engine based on Ray, to help build your own federated learning frameworks in minutes.☆97Aug 28, 2024Updated 2 years ago
- CUPTI GPU Profiler☆39Feb 26, 2019Updated 7 years ago
- Large language model fine-tuning capabilities based on cloud native and distributed computing.☆93Feb 22, 2024Updated 2 years ago
- Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)☆211Updated this week
- HeliosArtifact☆22Sep 27, 2022Updated 4 years ago
- ☆10Mar 13, 2023Updated 3 years ago
- Training and serving large-scale neural networks with auto parallelization.☆3,177Dec 9, 2023Updated 2 years ago
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆291Feb 2, 2026Updated 9 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Serving multiple LoRA finetuned LLM as one☆1,177May 8, 2024Updated 2 years ago
- Large Language Model Text Generation Inference on Habana Gaudi☆34Mar 20, 2025Updated last year
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆126Mar 13, 2024Updated 2 years ago
- oneAPI Collective Communications Library (oneCCL)☆273Updated this week
- Framework to achieve context distillation in LLMs☆15Nov 24, 2023Updated 2 years ago
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆228Oct 14, 2026Updated 3 weeks ago
- Machine Learning Inference Graph Spec☆21Jul 27, 2019Updated 7 years ago