dagster-io / dagster
An orchestration platform for the development, production, and observation of data assets.
☆12,772Updated this week
Alternatives and similar repositories for dagster:
Users that are interested in dagster are comparing it to the libraries listed below
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆18,661Updated this week
- dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build application…☆10,554Updated this week
- Always know what to expect from your data.☆10,273Updated this week
- Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting…☆4,534Updated last week
- Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.☆6,109Updated this week
- the portable Python dataframe library☆5,608Updated this week
- The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lak…☆17,591Updated this week
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆39,217Updated this week
- data load tool (dlt) is an open source Python library that makes data loading easy 🛠️☆3,355Updated this week
- Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to wr…☆2,001Updated this week
- Efficient data transformation and modeling framework that is backwards compatible with dbt.☆2,180Updated this week
- Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io☆2,046Updated this week
- Dataframes powered by a multithreaded, vectorized query engine, written in Rust☆32,499Updated this week
- A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rew…☆2,055Updated 6 months ago
- Build, Deploy and Manage AI/ML Systems☆8,644Updated this week
- OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata rep…☆6,310Updated this week
- Collect, aggregate, and visualize a data ecosystem's metadata☆1,879Updated this week
- Compare tables within or across databases☆2,964Updated 10 months ago
- A modular SQL linter and auto-formatter with support for multiple dialects and templated code.☆8,682Updated this week
- The Open Source Feature Store for AI/ML☆5,874Updated this week
- An Open Standard for lineage metadata collection☆1,879Updated this week
- Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and…☆10,228Updated this week
- The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-host…☆2,023Updated this week
- Dynamically generate Apache Airflow DAGs from YAML configuration files☆1,269Updated this week
- A light-weight, flexible, and expressive statistical data testing library☆3,688Updated 2 weeks ago
- Parallel computing with task scheduling☆13,045Updated this week
- Querybook is a Big Data Querying UI, combining collocated table metadata and a simple notebook interface.☆2,063Updated this week
- Distributed data engine for Python/SQL designed for the cloud, powered by Rust☆2,633Updated this week
- Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, v…☆4,281Updated this week
- re_data - fix data issues before your users & CEO would discover them 😊☆1,565Updated 10 months ago