spotify / luigiLinks
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,343Updated last month
Alternatives and similar repositories for luigi
Users that are interested in luigi are comparing it to the libraries listed below
Sorting:
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆40,585Updated this week
- Parallel computing with task scheduling☆13,277Updated this week
- Python composable command line interface toolkit☆16,523Updated this week
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆19,550Updated this week
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,390Updated 3 months ago
- Simple job queues for Python☆10,209Updated this week
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆27,707Updated 2 weeks ago
- Data-Centric Pipelines and Data Versioning☆6,234Updated 4 months ago
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,887Updated 11 months ago
- Faker is a Python package that generates fake data for you.☆18,468Updated last week
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,522Updated 9 months ago
- Serverless Python☆11,888Updated 2 years ago
- Python datetimes made easy☆6,462Updated last month
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,393Updated 8 months ago
- A formatter for Python files☆13,910Updated last month
- SQL for Humans™☆7,204Updated 11 months ago
- The property-based testing library for Python☆7,884Updated this week
- 📚 Parameterize, execute, and analyze notebooks☆6,197Updated 2 months ago
- Distributed Task Queue (development branch)☆26,612Updated this week
- An orchestration platform for the development, production, and observation of data assets.☆13,408Updated this week
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆23,664Updated this week
- Fuzzy String Matching in Python☆9,262Updated 2 years ago
- Python Development Workflow for Humans.☆25,062Updated last week
- Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk☆13,808Updated 10 months ago
- Interactive Data Visualization in the browser, from Python☆19,923Updated last week
- The no-magic web API and microservices framework for Python developers, with an emphasis on reliability and performance at scale.☆9,670Updated last week
- Accelerate your web app development | Build fast. Run fast.☆18,407Updated 2 months ago
- 🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library.…☆6,633Updated 3 months ago
- Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.☆19,317Updated 3 weeks ago
- Computing with Python functions.☆4,094Updated 3 weeks ago