spotify / luigiLinks
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,585Updated 6 months ago
Alternatives and similar repositories for luigi
Users that are interested in luigi are comparing it to the libraries listed below
Sorting:
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆43,383Updated this week
- Data-Centric Pipelines and Data Versioning☆6,268Updated 9 months ago
- Serverless Python☆11,861Updated 2 years ago
- Python Stream Processing☆6,824Updated last year
- Data Apps & Dashboards for Python. No JavaScript Required.☆24,296Updated this week
- Parallel computing with task scheduling☆13,627Updated last week
- Python composable command line interface toolkit☆17,023Updated last week
- 📚 Parameterize, execute, and analyze notebooks☆6,325Updated last week
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,536Updated last year
- Distributed Task Queue (development branch)☆27,648Updated this week
- Simple job queues for Python☆10,477Updated last week
- The no-magic web API and microservices framework for Python developers, with a focus on reliability and performance at scale.☆9,766Updated last week
- Python datetimes made easy☆6,592Updated last month
- Multi-user server for Jupyter notebooks☆8,187Updated this week
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆24,372Updated 2 weeks ago
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,482Updated last month
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆20,911Updated last week
- Python Development Workflow for Humans.☆25,121Updated last week
- Declarative visualization library for Python☆10,118Updated last week
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆28,003Updated last month
- NumPy and Pandas interface to Big Data☆3,199Updated 2 years ago
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,454Updated last month
- Python Serverless Microframework for AWS☆11,071Updated 6 months ago
- Machine Learning Toolkit for Kubernetes☆15,306Updated 3 weeks ago
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,914Updated last year
- A set of tools to keep your pinned Python dependencies fresh.☆7,972Updated this week
- Optional static typing for Python☆20,041Updated this week
- Ultra fast asyncio event loop.☆11,464Updated last month
- Accelerate your web app development | Build fast. Run fast.☆18,580Updated 2 months ago
- 🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library.…☆6,788Updated last month