apache / incubator-liminal
Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.
☆144Updated 8 months ago
Alternatives and similar repositories for incubator-liminal:
Users that are interested in incubator-liminal are comparing it to the libraries listed below
- The Internals of Spark on Kubernetes☆70Updated 2 years ago
- A Spark UI and Spark History Server alternative with CPU and Memory metrics! Delight is free, cross-platform, and open-source.☆345Updated 9 months ago
- Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pi…☆94Updated 2 weeks ago
- Spark-Dashboard is a solution for monitoring Apache Spark jobs. This repository provides the tooling and configuration for deploying an A…☆120Updated 3 weeks ago
- Performance optimization for Spark running on Kubernetes☆87Updated 4 years ago
- The Internals of Delta Lake☆183Updated 2 months ago
- A library that provides useful extensions to Apache Spark and PySpark.☆219Updated last week
- General Metadata Architecture☆124Updated this week
- ☆79Updated last year
- REST API for Apache Spark on K8S or YARN☆95Updated this week
- DynoYARN is a framework to run simulated YARN clusters and workloads for YARN scale testing.☆59Updated 2 years ago
- The Workload Analyzer collects Presto® and Trino workload statistics, and analyzes them☆135Updated last year
- ☆55Updated last year
- Example for article Running Spark 3 with standalone Hive Metastore 3.0☆97Updated 2 years ago
- Spline agent for Apache Spark☆191Updated 3 weeks ago
- The gateway component to make Spark on K8s much easier for Spark users.☆186Updated last month
- A portable Pythonic Data Lakehouse powered by Ray that brings exabyte-level scalability and fast, ACID-compliant, change-data-capture to …☆198Updated this week
- Smart Automation Tool for building modern Data Lakes and Data Pipelines☆120Updated this week
- Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are in…☆88Updated 11 months ago
- A library that brings useful functions from various modern database management systems to Apache Spark☆58Updated last year
- A simple Spark-powered ETL framework that just works 🍺☆181Updated 3 weeks ago
- A Table format agnostic data sharing framework☆38Updated last year
- CLI tool to bulk migrate the tables from one catalog another without a data copy☆76Updated 3 weeks ago
- A framework for writing performant user-defined functions (UDFs) that are portable across a variety of engines including Apache Spark, Ap…☆299Updated last year
- A temporary home for LinkedIn's changes to Apache Iceberg (incubating)☆62Updated 3 months ago
- Superglue is a lineage-tracking tool built to help visualize the propagation of data through complex pipelines composed of tables, jobs …☆157Updated 2 years ago
- Spark-Radiant is Apache Spark Performance and Cost Optimizer☆25Updated 2 months ago
- Spark on Kubernetes infrastructure Helm charts repo☆198Updated 2 years ago
- Performance Observability for Apache Spark☆235Updated this week
- Collection of open-source Spark tools & frameworks that have made the data engineering and data science teams at Swoop highly productive☆185Updated 2 years ago