spotify / luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
☆18,186Updated 2 months ago
Alternatives and similar repositories for luigi:
Users that are interested in luigi are comparing it to the libraries listed below
- Data-Centric Pipelines and Data Versioning☆6,214Updated last month
- Parallel computing with task scheduling☆13,083Updated last week
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows☆39,379Updated this week
- Data Apps & Dashboards for Python. No JavaScript Required.☆22,225Updated this week
- Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.☆19,022Updated 5 months ago
- Serverless Python☆11,884Updated 2 years ago
- Simple job queues for Python☆10,093Updated last week
- Accelerate your web app development | Build fast. Run fast.☆18,301Updated this week
- Machine Learning Toolkit for Kubernetes☆14,811Updated this week
- The no-magic web API and microservices framework for Python developers, with an emphasis on reliability and performance at scale.☆9,619Updated this week
- Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.☆6,879Updated 8 months ago
- Python composable command line interface toolkit☆16,201Updated 2 weeks ago
- Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.☆27,500Updated this week
- A next-generation curated knowledge sharing platform for data scientists and other technical professions.☆5,508Updated 6 months ago
- Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.☆27,136Updated this week
- Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per s…☆8,357Updated 5 months ago
- Visualizations for machine learning datasets☆7,366Updated last year
- A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C proje…☆23,272Updated last week
- 📚 Parameterize, execute, and analyze notebooks☆6,115Updated 2 months ago
- Write scalable load tests in plain Python 🚗💨☆25,902Updated this week
- A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.☆9,874Updated last month
- Python job scheduling for humans.☆12,012Updated 10 months ago
- A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin☆6,322Updated 3 weeks ago
- PredictionIO, a machine learning server for developers and ML engineers.☆12,529Updated 4 years ago
- Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk☆13,634Updated 8 months ago
- Ready-to-run Docker images containing Jupyter applications☆8,156Updated this week
- Declarative visualization library for Python☆9,673Updated 3 weeks ago
- Python Stream Processing☆6,777Updated 8 months ago
- Prefect is a workflow orchestration framework for building resilient data pipelines in Python.☆18,797Updated this week
- Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.☆36,271Updated this week