spotify / luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,234Updated 2 months ago
Alternatives and similar repositories for luigi:
Users that are interested in luigi are comparing it to the libraries listed below
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆39,775Updated this week
- A formatter for Python files☆13,880Updated 3 weeks ago
- Parallel computing with task scheduling☆13,136Updated last week
- Data-Centric Pipelines and Data Versioning☆6,222Updated 2 months ago
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,881Updated 9 months ago
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆19,062Updated this week
- 🦉 Data Versioning and ML Experiments☆14,402Updated this week
- The property-based testing library for Python☆7,813Updated this week
- Build, Manage and Deploy AI/ML Systems☆8,742Updated this week
- Python composable command line interface toolkit☆16,270Updated 2 weeks ago
- Data Apps & Dashboards for Python. No JavaScript Required.☆22,337Updated this week
- A very fast and expressive template engine.☆10,796Updated last month
- Accelerate your web app development | Build fast. Run fast.☆18,346Updated 3 weeks ago
- Ready-to-run Docker images containing Jupyter applications☆8,185Updated this week
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,371Updated 6 months ago
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,513Updated 7 months ago
- The no-magic web API and microservices framework for Python developers, with an emphasis on reliability and performance at scale.☆9,639Updated this week
- 💫 Industrial-strength Natural Language Processing (NLP) in Python☆31,454Updated 2 weeks ago
- Ultra fast asyncio event loop.☆10,825Updated last week
- 📚 Parameterize, execute, and analyze notebooks☆6,137Updated 2 weeks ago
- Serverless Python☆11,889Updated 2 years ago
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆27,586Updated last week
- the portable Python dataframe library☆5,705Updated this week
- Modin: Scale your Pandas workflows by changing a single line of code☆10,116Updated last week
- Docker Apache Airflow☆3,803Updated 2 years ago
- Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.☆19,129Updated 5 months ago
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆23,415Updated this week
- Computing with Python functions.☆4,046Updated this week
- Interactive Data Visualization in the browser, from Python☆19,785Updated this week
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,346Updated last month