DavidTorpey / pydags
Simple, lightweight, extensible DAG framework for Python with a Kubeflow-like API
☆78Updated last year
Alternatives and similar repositories for pydags:
Users that are interested in pydags are comparing it to the libraries listed below
- Shared-memory Python object namespace with Apache Plasma. Built because of Plotly Dash, useful anywhere.☆84Updated 2 months ago
- Stream Processing Made Easy☆40Updated 2 years ago
- A collection of python utility functions☆11Updated 8 months ago
- Distributed Task Queue based Dask☆38Updated last year
- A robust DAG implementation for parallel execution☆67Updated last year
- Tutorials for Fugue - A unified interface for distributed computing. Fugue executes SQL, Python, and Pandas code on Spark and Dask withou…☆113Updated 11 months ago
- Concurrent appendable key-value storage☆105Updated 8 months ago
- A Python package that parses SQL and interprets it as methods that act upon existing pandas (or other types of) DataFrames that have been…☆98Updated 3 years ago
- Swiple enables you to easily observe, understand, validate and improve the quality of your data☆82Updated this week
- Define your JSON schema as Python dataclasses☆63Updated last year
- Function dependencies resolution and execution☆71Updated 4 years ago
- Unified Distributed Execution☆51Updated 5 months ago
- An easy to use rpc framework for enabling fast inter-process, inter-container, or inter-host communication☆65Updated 2 years ago
- A hierarchical, schema-based solution for fetching and crafting data, from simple to complicated.☆38Updated 4 months ago
- 🦖 A SQL-on-everything Query Engine you can execute over multiple databases and file formats. Query your data, where it lives.☆101Updated this week
- Simple Workflow Framework - Hamilton + APScheduler = FlowerPower☆15Updated this week
- Operator overloading for fast anonymous functions.☆50Updated 3 weeks ago
- A small Python module containing quick utility functions for standard ETL processes.☆34Updated this week
- dagster scikit-learn pipeline example.☆45Updated 2 years ago
- A framework for piping in python.☆73Updated 4 years ago
- Example Polylith setup for Python☆52Updated 2 months ago
- A collection of self-contained fsspec-based filesystems☆15Updated last week
- A general-purpose library designed to guide developers in expressing their code as a flow.☆101Updated last month
- Meatie is a Python metaprogramming library that eliminates the need for boilerplate code when integrating with REST APIs. Meatie abstrac…☆52Updated this week
- Vinum is a SQL processor for Python, designed for data analysis workflows and in-memory analytics.☆65Updated 3 years ago
- Multiprocessing Queues Made Easy☆111Updated 3 months ago
- A purely-functional HTML builder for Python. Think JSX rather than templates.☆96Updated 2 months ago
- simple, flexible, offline capable, cloud storage with a Python path-like interface☆173Updated 9 months ago
- YAML support for Pydantic models☆163Updated this week
- Read Apache Arrow batches from ODBC data sources in Python☆65Updated last week