spotify / luigiLinks
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,349Updated last month
Alternatives and similar repositories for luigi
Users that are interested in luigi are comparing it to the libraries listed below
Sorting:
- Parallel computing with task scheduling☆13,302Updated last week
- Python Stream Processing☆6,798Updated 11 months ago
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆40,757Updated this week
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,888Updated 11 months ago
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,390Updated 3 months ago
- 📚 Parameterize, execute, and analyze notebooks☆6,201Updated 2 months ago
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,523Updated 9 months ago
- A formatter for Python files☆13,914Updated this week
- The property-based testing library for Python☆7,896Updated this week
- Python datetimes made easy☆6,475Updated last week
- Build, Manage and Deploy AI/ML Systems☆8,913Updated this week
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆23,717Updated last week
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆19,658Updated this week
- Simple job queues for Python☆10,224Updated last week
- A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow☆2,079Updated last year
- Python Development Workflow for Humans.☆25,068Updated last week
- Distributed Task Queue (development branch)☆26,700Updated this week
- Data Apps & Dashboards for Python. No JavaScript Required.☆23,217Updated this week
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆27,724Updated last month
- Accelerate your web app development | Build fast. Run fast.☆18,421Updated this week
- Data-Centric Pipelines and Data Versioning☆6,236Updated 4 months ago
- 🦉 Data Versioning and ML Experiments☆14,595Updated this week
- Docker Apache Airflow☆3,806Updated 2 years ago
- The no-magic web API and microservices framework for Python developers, with an emphasis on reliability and performance at scale.☆9,676Updated this week
- Declarative visualization library for Python☆9,862Updated last month
- Python Serverless Microframework for AWS☆10,879Updated last month
- GraphQL framework for Python☆8,203Updated 7 months ago
- Python composable command line interface toolkit☆16,564Updated this week
- Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.☆19,351Updated last month
- 🏹 Better dates & times for Python☆8,874Updated last month