spotify / luigiLinks
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,477Updated 3 months ago
Alternatives and similar repositories for luigi
Users that are interested in luigi are comparing it to the libraries listed below
Sorting:
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆42,236Updated last week
- Parallel computing with task scheduling☆13,481Updated this week
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,441Updated 3 weeks ago
- Data-Centric Pipelines and Data Versioning☆6,252Updated 7 months ago
- Python Stream Processing☆6,822Updated last year
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆20,315Updated this week
- Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics☆15,929Updated this week
- Build, Manage and Deploy AI/ML Systems☆9,453Updated last week
- Serverless Python☆11,876Updated 2 years ago
- A curated list of awesome ETL frameworks, libraries, and software.☆3,456Updated last year
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,894Updated last year
- Data Apps & Dashboards for Python. No JavaScript Required.☆24,021Updated last week
- Ultra fast asyncio event loop.☆11,250Updated 4 months ago
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,428Updated last week
- Python Development Workflow for Humans.☆25,096Updated 2 months ago
- the portable Python dataframe library☆6,094Updated this week
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆27,865Updated 2 weeks ago
- Screaming-fast Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser.☆8,578Updated 2 years ago
- Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.☆27,751Updated this week
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,527Updated last year
- Interactive Data Visualization in the browser, from Python☆20,094Updated this week
- A formatter for Python files☆13,948Updated last week
- Declarative visualization library for Python☆9,997Updated last week
- 📘 The interactive computing suite for you! ✨☆6,260Updated last year
- Utils for streaming large files (S3, HDFS, gzip, bz2...)☆3,361Updated this week
- An orchestration platform for the development, production, and observation of data assets.☆13,990Updated this week
- The property-based testing library for Python☆8,065Updated last week
- ReactiveX for Python☆4,947Updated 3 months ago
- Multi-user server for Jupyter notebooks☆8,119Updated this week
- NumPy and Pandas interface to Big Data☆3,198Updated last year