giacbrd / SmartPipelineLinks
A framework for rapid development of robust data pipelines following a simple design pattern
☆27Updated last year
Alternatives and similar repositories for SmartPipeline
Users that are interested in SmartPipeline are comparing it to the libraries listed below
Sorting:
- ☆36Updated 5 months ago
- Curated list of awesome software and resources for Senzing, The First Real-Time AI for Entity Resolution.☆59Updated this week
- dagster scikit-learn pipeline example.☆44Updated 2 years ago
- Library of Prefect tasks and utilities.☆9Updated 9 months ago
- Graph Engine for Exploration and Search☆42Updated last year
- Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given website☆44Updated 7 years ago
- A small Python module containing quick utility functions for standard ETL processes.☆36Updated last week
- Python ELT Studio, an application for building ELT (and ETL) data flows.☆58Updated 3 years ago
- dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.☆57Updated 3 years ago
- Simple, lightweight, extensible DAG framework for Python with a Kubeflow-like API☆80Updated last year
- Python package for deduplication/entity resolution using active learning☆81Updated 10 months ago
- Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).☆120Updated last month
- Light weight labeling engine☆12Updated 3 years ago
- A curated list of dagster code snippets for data engineers☆56Updated last year
- Distributed persistent Task Queue running on Dask☆38Updated 2 years ago
- Beneath is a serverless real-time data platform ⚡️☆84Updated 3 years ago
- DataForge helps data teams write functional transformation pipelines by leveraging software engineering principles☆51Updated 2 weeks ago
- Fuzzy matching for companies'names☆9Updated 5 years ago
- A helm chart for Prefect☆14Updated 5 years ago
- Kubetools is a tool and processes for developing and deploying microservices to Kubernetes.☆14Updated 7 months ago
- This repository is part of an article "Prefect workflow automation with Azure DevOps and AKS"☆30Updated 4 years ago
- Create and manage data pipes with Meerschaum.☆143Updated this week
- Text Processing & Segmentation Framework☆23Updated 3 months ago
- Versatile Metrics Collection for Python☆19Updated last year
- Ssebowa is free and open source library in Python that provides generative-ai models.☆14Updated last year
- Async bulk data ingestion and querying in various document, graph and vector databases via their Python clients☆36Updated last year
- ☆11Updated 2 years ago
- MLflow and Prefect with docker-compose☆16Updated 2 years ago
- A fully-featured multi-source data pipeline for continuously extracting knowledge from COVID-19 data.☆21Updated 4 years ago
- Entity resolution, also known as Data Matching or Record linkage is the task of finding a data set that refer to the same or similar real…☆24Updated 3 months ago