spotify / luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,095Updated 2 weeks ago
Alternatives and similar repositories for luigi:
Users that are interested in luigi are comparing it to the libraries listed below
- Serverless Python☆11,880Updated last year
- Python composable command line interface toolkit☆16,055Updated 3 weeks ago
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,870Updated 7 months ago
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆23,074Updated last week
- Parallel computing with task scheduling☆12,934Updated this week
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆38,743Updated this week
- Data-Centric Pipelines and Data Versioning☆6,203Updated 2 weeks ago
- Utils for streaming large files (S3, HDFS, gzip, bz2...)☆3,265Updated 2 months ago
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆27,409Updated 2 weeks ago
- Python logging made (stupidly) simple☆20,816Updated this week
- Declarative visualization library for Python☆9,582Updated last week
- Python datetimes made easy☆6,359Updated 2 weeks ago
- Machine Learning Toolkit for Kubernetes☆14,651Updated this week
- Hypothesis is a powerful, flexible, and easy to use library for property-based testing.☆7,705Updated this week
- Python Development Workflow for Humans.☆24,980Updated this week
- Grumpy is a Python to Go source code transcompiler and runtime.☆10,543Updated 3 years ago
- The no-magic web API and microservices framework for Python developers, with an emphasis on reliability and performance at scale.☆9,599Updated last week
- Sampling profiler for Python programs☆13,249Updated last week
- Ultra fast asyncio event loop.☆10,651Updated 4 months ago
- Open Source AI/ML Platform☆8,535Updated this week
- 📚 Parameterize, execute, and analyze notebooks☆6,080Updated last month
- Python packaging and dependency management made easy☆32,578Updated this week
- 🏹 Better dates & times for Python☆8,792Updated 2 months ago
- REST API framework designed for human beings☆6,712Updated last month
- Simple, Pythonic remote execution and deployment.☆15,003Updated 10 months ago
- A Python library that generates static type annotations by collecting runtime types☆4,847Updated 7 months ago
- Multi-user server for Jupyter notebooks☆7,908Updated this week
- gunicorn 'Green Unicorn' is a WSGI HTTP Server for UNIX, fast clients and sleepy applications.☆9,982Updated 3 months ago
- Network Analysis in Python☆15,409Updated this week
- Library for building powerful interactive command line applications in Python☆9,538Updated 3 weeks ago