Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,781Oct 14, 2026Updated 2 weeks ago
Alternatives and similar repositories for luigi
Users that are interested in luigi are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆47,333Updated this week
- Parallel computing with task scheduling☆13,941Sep 29, 2026Updated last month
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆24,103Updated this week
- Apache Superset is a Data Visualization and Data Exploration Platform☆75,381Updated this week
- Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.☆28,855Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆28,225Jul 1, 2026Updated 4 months ago
- Distributed Task Queue (development branch)☆29,096Updated this week
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆25,175Apr 1, 2026Updated 7 months ago
- Pinball is a scalable workflow manager☆1,046Dec 10, 2019Updated 6 years ago
- Interactive Data Visualization in the browser, from Python☆20,468Updated this week
- Simple job queues for Python☆10,701Updated this week
- Data-Centric Pipelines and Data Versioning☆6,313Feb 3, 2025Updated last year
- Python Development Workflow for Humans.☆25,019Updated this week
- An orchestration platform for the development, production, and observation of data assets.☆16,357Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Python Stream Processing☆6,823Jul 27, 2024Updated 2 years ago
- Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk☆14,324Oct 29, 2025Updated last year
- 💫 Industrial-strength Natural Language Processing (NLP) in Python☆34,005Updated this week
- Serverless Python☆11,811Mar 23, 2023Updated 3 years ago
- The no-magic web API and microservices framework for Python developers, with a focus on reliability and performance at scale.☆9,811Oct 24, 2026Updated last week
- Accelerate your web app development | Build fast. Run fast.☆18,632Jul 29, 2026Updated 3 months ago
- A formatter for Python files☆13,987Updated this week
- Data Apps & Dashboards for Python. No JavaScript Required.☆24,474Updated this week
- Python composable command line interface toolkit☆17,801Oct 6, 2026Updated 3 weeks ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,629Oct 11, 2026Updated 3 weeks ago
- Python packaging and dependency management made easy☆34,326Updated this week
- 🏹 Better dates & times for Python☆9,054Jun 22, 2026Updated 4 months ago
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,873Jul 4, 2024Updated 2 years ago
- Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.☆20,468Oct 22, 2026Updated last week
- Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.☆44,171Updated this week
- Simple, Pythonic remote execution and deployment.☆15,520Apr 10, 2026Updated 6 months ago
- A collection of design patterns/idioms in Python☆43,077Updated this week
- 🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library.…☆6,940Oct 25, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Write scalable load tests in plain Python 🚗💨☆28,254Updated this week
- The uncompromising Python code formatter☆42,085Updated this week
- NumPy and Pandas interface to Big Data☆3,187Sep 29, 2023Updated 3 years ago
- 🦉 Data Versioning and ML Experiments☆15,955Updated this week
- Python job scheduling for humans.☆12,267May 25, 2024Updated 2 years ago
- A simple, yet elegant, HTTP library.☆54,669Updated this week
- Ultra fast asyncio event loop.☆11,927Oct 19, 2026Updated last week