PrefectHQ / prefect
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
☆19,009Updated this week
Alternatives and similar repositories for prefect:
Users that are interested in prefect are comparing it to the libraries listed below
- An orchestration platform for the development, production, and observation of data assets.☆12,990Updated this week
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆39,674Updated this week
- the portable Python dataframe library☆5,699Updated this week
- Dataframes powered by a multithreaded, vectorized query engine, written in Rust☆33,170Updated this week
- dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build application…☆10,685Updated this week
- Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, vis…☆18,234Updated 2 months ago
- Always know what to expect from your data.☆10,334Updated this week
- Parallel computing with task scheduling☆13,136Updated this week
- The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lak…☆17,903Updated this week
- Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.☆6,183Updated this week
- Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and…☆10,276Updated this week
- DuckDB is an analytical in-process SQL database management system☆28,513Updated this week
- Python packaging and dependency management made easy☆33,037Updated this week
- Build, Manage and Deploy AI/ML Systems☆8,730Updated this week
- A modular SQL linter and auto-formatter with support for multiple dialects and templated code.☆8,788Updated this week
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,366Updated 6 months ago
- Data validation using Python type hints☆23,344Updated this week
- Modin: Scale your Pandas workflows by changing a single line of code☆10,116Updated this week
- 🦉 Data Versioning and ML Experiments☆14,381Updated last week
- A light-weight, flexible, and expressive statistical data testing library☆3,754Updated this week
- Build data pipelines, the easy way 🛠️☆4,113Updated last year
- Streamlit — A faster way to build and share data apps.☆38,865Updated this week
- Pyodide is a Python distribution for the browser and Node.js based on WebAssembly☆13,042Updated last week
- 📚 Parameterize, execute, and analyze notebooks☆6,137Updated 2 weeks ago
- Python Stream Processing☆6,786Updated 8 months ago
- Python logging made (stupidly) simple☆21,428Updated 3 weeks ago
- Diagram as Code for prototyping cloud system architectures☆40,620Updated last week
- The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!☆7,629Updated this week
- 🧙 Build, run, and manage data pipelines for integrating and transforming data.☆8,262Updated last week
- An open-source runtime for composable workflows. Great for AI agents and CI/CD.☆13,572Updated this week