RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.
☆368Feb 1, 2026Updated last month
Alternatives and similar repositories for raydp
Users that are interested in raydp are comparing it to the libraries listed below
Sorting:
- Mobius is an AI infrastructure platform for distributed online learning, including online sample processing, training and serving.☆100Jun 21, 2024Updated last year
- A portable Multimodal Lakehouse powered by Ray that brings exabyte-level scalability and fast, ACID-compliant, change-data-capture to you…☆268Feb 18, 2026Updated last week
- A toolkit to run Ray applications on Kubernetes☆2,341Updated this week
- A Ray-based data loader with per-epoch shuffling and configurable pipelining, for shuffling and loading training data for distributed tra…☆18Jan 5, 2023Updated 3 years ago
- WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.☆17Aug 4, 2022Updated 3 years ago
- Distributed SQL Query Engine in Python using Ray☆245Oct 2, 2024Updated last year
- Spark RAPIDS plugin - accelerate Apache Spark with GPUs☆963Updated this week
- Some microbenchmarks and design docs before commencement☆12Feb 1, 2021Updated 5 years ago
- Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.☆1,516Updated this week
- Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.☆257Feb 21, 2023Updated 3 years ago
- BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray☆2,692Updated this week
- MLFlow Deployment Plugin for Ray Serve☆46Apr 12, 2022Updated 3 years ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆130Sep 23, 2025Updated 5 months ago
- Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.☆41,516Updated this week
- Distributed XGBoost on Ray☆154Jun 25, 2024Updated last year
- Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. AntRay i…☆165Feb 14, 2026Updated 2 weeks ago
- A composable and fully extensible C++ execution engine library for data management systems.☆4,065Updated this week
- Rayvens makes it possible for data scientists to access hundreds of data services within Ray with little effort.☆50Nov 29, 2022Updated 3 years ago
- High performance model preprocessing library on PyTorch☆646Mar 29, 2024Updated last year
- Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.☆2,749Jan 2, 2024Updated 2 years ago
- Pytorch Lightning Distributed Accelerators using Ray☆216Nov 3, 2023Updated 2 years ago
- Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet f…☆1,879Jan 2, 2026Updated last month
- Exoshuffle-CloudSort☆29Mar 2, 2023Updated 3 years ago
- RayLLM - LLMs on Ray (Archived). Read README for more info.☆1,267Mar 13, 2025Updated 11 months ago
- High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale☆5,250Updated this week
- Ray tutorials from Anyscale☆631Nov 9, 2023Updated 2 years ago
- Ray-based Apache Beam runner☆42Aug 30, 2023Updated 2 years ago
- Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.☆1,039Updated this week
- Deep Learning on Flink aims to integrate Flink and deep learning frameworks (e.g. TensorFlow, PyTorch, etc) to enable distributed deep le…☆695Nov 12, 2024Updated last year
- Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.☆3,106Updated this week
- A Spark UI and Spark History Server alternative with CPU and Memory metrics! Delight is free, cross-platform, and open-source.☆347May 31, 2024Updated last year
- The Auron accelerator for distributed computing framework (e.g., Spark) leverages native vectorized execution to accelerate query process…☆1,715Updated this week
- A runtime implementation of data-parallel actors.☆39Apr 1, 2022Updated 3 years ago
- Resource-adaptive cluster scheduler for deep learning training.☆454Mar 5, 2023Updated 2 years ago
- Byzer (former MLSQL): A low-code open-source programming language for data pipeline, analytics and AI.☆1,845May 29, 2024Updated last year
- A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rew…☆2,139Feb 21, 2026Updated last week
- Ray provider for Apache Airflow☆47Feb 7, 2024Updated 2 years ago
- Deadline-based hyperparameter tuning on RayTune.☆32Jan 16, 2020Updated 6 years ago
- Tracking Ray Enhancement Proposals☆65Dec 17, 2025Updated 2 months ago