spotify / luigiLinks
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,609Updated 7 months ago
Alternatives and similar repositories for luigi
Users that are interested in luigi are comparing it to the libraries listed below
Sorting:
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆43,549Updated last week
- Parallel computing with task scheduling☆13,657Updated this week
- Data-Centric Pipelines and Data Versioning☆6,275Updated 10 months ago
- Python Stream Processing☆6,830Updated last year
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,490Updated last month
- 🦉 Data Versioning and ML Experiments☆15,202Updated last week
- 📚 Parameterize, execute, and analyze notebooks☆6,340Updated 3 weeks ago
- Data Apps & Dashboards for Python. No JavaScript Required.☆24,351Updated this week
- A collection of design patterns/idioms in Python☆42,536Updated 3 weeks ago
- Computing with Python functions.☆4,294Updated last week
- The property-based testing library for Python☆8,312Updated last week
- Modin: Scale your Pandas workflows by changing a single line of code☆10,340Updated 2 months ago
- Python datetimes made easy☆6,599Updated last week
- Distributed Task Queue (development branch)☆27,753Updated this week
- NumPy and Pandas interface to Big Data☆3,198Updated 2 years ago
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆24,458Updated this week
- An orchestration platform for the development, production, and observation of data assets.☆14,618Updated this week
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆28,031Updated last month
- 🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library.…☆6,808Updated 2 months ago
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,537Updated last year
- Python composable command line interface toolkit☆17,061Updated last week
- Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.☆6,639Updated this week
- Python packaging and dependency management made easy☆34,129Updated last week
- Extract Transform Load for Python 3.5+☆1,604Updated 2 years ago
- Apache Superset is a Data Visualization and Data Exploration Platform☆69,451Updated this week
- the portable Python dataframe library☆6,282Updated last week
- Utils for streaming large files (S3, HDFS, gzip, bz2...)☆3,419Updated 3 weeks ago
- Grumpy is a Python to Go source code transcompiler and runtime.☆10,520Updated 3 years ago
- Build, Manage and Deploy AI/ML Systems☆9,678Updated this week
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,463Updated last month