pydiverse / pydiverse.pipedagLinks
A data pipeline orchestration library for rapid iterative development with automatic cache invalidation allowing users to focus writing their tasks in pandas, polars, sqlalchemy, ibis, and alike.
☆34Updated this week
Alternatives and similar repositories for pydiverse.pipedag
Users that are interested in pydiverse.pipedag are comparing it to the libraries listed below
Sorting:
- RFC document, tooling and other content related to the dataframe API standard☆110Updated last year
- Coming soon☆61Updated last year
- IbisML is a library for building scalable ML pipelines using Ibis.☆113Updated last week
- Dask integration for Snowflake☆30Updated last week
- Assessing whether data from database complies with reference information.☆43Updated last week
- A toolbox 🧰 for Jupyter notebooks 📙: testing, experiment tracking, debugging, profiling, and more!☆67Updated 10 months ago
- A repository of runnable examples using ibis☆44Updated last year
- A data modelling layer built on top of polars and pydantic☆198Updated 2 years ago
- Arrow, pydantic style☆84Updated 2 years ago
- ☆38Updated this week
- Capture all information throughout your model's development in a reproducible way and tie results directly to the model code!☆134Updated last week
- Run pytest against markdown files/docstrings.☆124Updated last week
- Extremely lightweight compatibility layer between pandas and Polars☆41Updated last year
- Python bindings and arrow integration for the rust object_store crate.☆63Updated 11 months ago
- Automatically upgrade your Polars code to use the latest syntax available☆65Updated last year
- Polars plugin for stable hashing functionality☆77Updated 3 months ago
- ☆89Updated 6 months ago
- Cloud provider cluster managers for Dask. Supports AWS, Google Cloud Azure and more...☆143Updated 3 weeks ago
- Typed wrappers over pandas DataFrames with schema validation☆102Updated last year
- fsspec-compatible Azure Datake and Azure Blob Storage access☆195Updated this week
- A multi-tenant server for securely deploying and managing Dask clusters.☆142Updated 3 weeks ago
- Easy and flexible data contracts☆155Updated last month
- Polars plugin offering eXtra stuff for DateTimes☆215Updated last week
- Python package implementing transformers for pre processing steps for machine learning.☆62Updated this week
- Kedro plugin to support running pipelines on Dagster☆13Updated last week
- A place to provide Coiled feedback☆20Updated 4 months ago
- An Python object protocol for projects to interchange data frame-like data without forcing pandas.DataFrame as the intermediary☆15Updated 5 years ago
- Polars extension for fzf-style fuzzy matching☆30Updated 11 months ago
- Automated, schema-based JSON unpacking to Polars objects☆13Updated last year
- A curated list of polars projects and resources.☆37Updated 4 months ago