spotify / luigiLinks
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,617Updated last week
Alternatives and similar repositories for luigi
Users that are interested in luigi are comparing it to the libraries listed below
Sorting:
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆43,722Updated this week
- Python Stream Processing☆6,831Updated last year
- Parallel computing with task scheduling☆13,679Updated last week
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆21,253Updated this week
- Data-Centric Pipelines and Data Versioning☆6,277Updated 11 months ago
- The property-based testing library for Python☆8,361Updated this week
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆24,479Updated 2 weeks ago
- 🦉 Data Versioning and ML Experiments☆15,249Updated this week
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,907Updated last year
- A formatter for Python files☆13,978Updated 3 weeks ago
- Distributed Task Queue (development branch)☆27,823Updated this week
- Serverless Python☆11,860Updated 2 years ago
- Interactive Data Visualization in the browser, from Python☆20,277Updated this week
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,502Updated 2 months ago
- Simple job queues for Python☆10,539Updated this week
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆28,048Updated 2 months ago
- the portable Python dataframe library☆6,318Updated last week
- The no-magic web API and microservices framework for Python developers, with a focus on reliability and performance at scale.☆9,777Updated last week
- 🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library.…☆6,816Updated last week
- 📚 Parameterize, execute, and analyze notebooks☆6,345Updated last month
- Declarative visualization library for Python☆10,196Updated last week
- Computing with Python functions.☆4,302Updated last week
- Python Development Workflow for Humans.☆25,108Updated 3 weeks ago
- Real-time monitor and web admin for Celery distributed task queue☆7,083Updated last month
- gunicorn 'Green Unicorn' is a WSGI HTTP Server for UNIX, fast clients and sleepy applications.☆10,390Updated last month
- Python logging made (stupidly) simple☆23,374Updated last week
- NumPy and Pandas interface to Big Data☆3,199Updated 2 years ago
- Python datetimes made easy☆6,601Updated 3 weeks ago
- Accelerate your web app development | Build fast. Run fast.☆18,609Updated last week
- Utils for streaming large files (S3, HDFS, gzip, bz2...)☆3,422Updated last week