spotify / luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,203Updated 2 months ago
Alternatives and similar repositories for luigi:
Users that are interested in luigi are comparing it to the libraries listed below
- Declarative visualization library for Python☆9,673Updated 3 weeks ago
- Parallel computing with task scheduling☆13,083Updated last week
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆39,458Updated this week
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆27,525Updated this week
- Data-Centric Pipelines and Data Versioning☆6,214Updated 2 months ago
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,322Updated 3 weeks ago
- Data Apps & Dashboards for Python. No JavaScript Required.☆22,248Updated this week
- Python Stream Processing☆6,777Updated 8 months ago
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,360Updated 5 months ago
- Python composable command line interface toolkit☆16,201Updated 2 weeks ago
- A formatter for Python files☆13,874Updated this week
- 🦉 Data Versioning and ML Experiments☆14,324Updated last week
- Build, Manage and Deploy AI/ML Systems☆8,689Updated this week
- Simple job queues for Python☆10,093Updated last week
- Interactive Data Visualization in the browser, from Python☆19,746Updated this week
- 📚 Parameterize, execute, and analyze notebooks☆6,115Updated 2 months ago
- Machine Learning Toolkit for Kubernetes☆14,811Updated last week
- Streamlit — A faster way to build and share data apps.☆38,572Updated this week
- Modin: Scale your Pandas workflows by changing a single line of code☆10,095Updated this week
- Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.☆27,136Updated this week
- Always know what to expect from your data.☆10,297Updated this week
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,879Updated 8 months ago
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆18,797Updated this week
- Ready-to-run Docker images containing Jupyter applications☆8,156Updated this week
- Computing with Python functions.☆4,022Updated last week
- Postgres CLI with autocompletion and syntax highlighting☆12,349Updated this week
- Python Development Workflow for Humans.☆25,021Updated this week
- the portable Python dataframe library☆5,648Updated this week
- A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow☆2,080Updated last year
- SQL for Humans™☆7,187Updated 8 months ago