jacopotagliabue / paas-data-ingestionLinks
Ingesting data with Pulumi, AWS lambdas and Snowflake in a scalable, fully replayable manner
☆71Updated 3 years ago
Alternatives and similar repositories for paas-data-ingestion
Users that are interested in paas-data-ingestion are comparing it to the libraries listed below
Sorting:
- Joining the modern data stack with the modern ML stack☆196Updated 2 years ago
- Playground for using large language models into the Modern Data Stack for entity matching☆108Updated 2 years ago
- Recommendations at "Reasonable Scale": joining dataOps with recSys through dbt, Merlin and Metaflow☆237Updated 2 years ago
- A PaaS End-to-End ML Setup with Metaflow, Serverless and SageMaker.☆37Updated 4 years ago
- Supporting materials/code examples for my course in data engineering for machine learning.☆38Updated 2 years ago
- Data pipeline with dbt, Airflow, Great Expectations☆163Updated 3 years ago
- Repo for orienting dbt users to the Dagster asset framework☆54Updated 2 years ago
- Kedro Plugin to support running workflows on Kubeflow Pipelines☆54Updated 9 months ago
- A simple and easy to use Data Quality (DQ) tool built with Python.☆50Updated last year
- A GitHub Action that makes it easy to use Great Expectations to validate your data pipelines in your CI workflows.☆80Updated last year
- A write-audit-publish implementation on a data lake without the JVM☆46Updated 10 months ago
- A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.☆184Updated last year
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆169Updated last year
- Parse dbt artifacts and search dbt models with Algolia☆52Updated 4 years ago
- Demo repository to lambda-fy your dbt runs☆11Updated last year
- Food for thoughts around data contracts☆25Updated 3 months ago
- A guide for leading a data (engineering) team☆64Updated last year
- Feast AWS guide using Redshift / Spectrum / DynamoDB to build a credit scoring model☆64Updated 3 years ago
- 🛠 Python project template with unit tests, code coverage, linting, type checking, Makefile wrapper, and GitHub Actions.☆147Updated last year
- A web extension to empower dbt users☆27Updated 2 years ago
- A playground for running duckdb as a stateless query engine over a data lake☆202Updated last year
- ☆80Updated 8 months ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆88Updated 4 years ago
- A dbt package for doing product analytics☆85Updated 2 years ago
- IbisML is a library for building scalable ML pipelines using Ibis.☆109Updated 5 months ago
- Tutorials for Fugue - A unified interface for distributed computing. Fugue executes SQL, Python, and Pandas code on Spark and Dask withou…☆113Updated last year
- ☆85Updated 2 years ago
- Pytest plugin for dbt core☆60Updated 5 months ago
- Delta Lake helper methods. No Spark dependency.☆23Updated 9 months ago
- Define, govern, and model event data for warehouse-first product analytics.☆83Updated 11 months ago