spotify / luigiLinks
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,635Updated this week
Alternatives and similar repositories for luigi
Users that are interested in luigi are comparing it to the libraries listed below
Sorting:
- Parallel computing with task scheduling☆13,729Updated last week
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆44,086Updated this week
- Data-Centric Pipelines and Data Versioning☆6,286Updated last year
- Modin: Scale your Pandas workflows by changing a single line of code☆10,351Updated 4 months ago
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,468Updated 3 months ago
- Python composable command line interface toolkit☆17,146Updated last month
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆24,610Updated this week
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,540Updated last year
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,518Updated 3 months ago
- 🦉 Data Versioning and ML Experiments☆15,308Updated last week
- Build, Manage and Deploy AI/ML Systems☆9,743Updated this week
- Python Stream Processing☆6,834Updated last year
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,905Updated last year
- Faker is a Python package that generates fake data for you.☆19,064Updated 3 weeks ago
- 📚 Parameterize, execute, and analyze notebooks☆6,360Updated 3 weeks ago
- the portable Python dataframe library☆6,371Updated this week
- The no-magic web API and microservices framework for Python developers, with a focus on reliability and performance at scale.☆9,785Updated last week
- Declarative visualization library for Python☆10,232Updated last week
- Distributed Task Queue (development branch)☆27,938Updated this week
- The property-based testing library for Python☆8,422Updated this week
- Computing with Python functions.☆4,321Updated 3 weeks ago
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆28,080Updated 3 months ago
- Python datetimes made easy☆6,611Updated 2 weeks ago
- A formatter for Python files☆13,978Updated last week
- Ready-to-run Docker images containing Jupyter applications☆8,402Updated last week
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆21,507Updated this week
- Data Apps & Dashboards for Python. No JavaScript Required.☆24,429Updated last week
- Simple job queues for Python☆10,557Updated last week
- Utils for streaming large files (S3, HDFS, gzip, bz2...)☆3,429Updated this week
- Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.☆28,193Updated this week