jacopotagliabue / paas-data-ingestionLinks
Ingesting data with Pulumi, AWS lambdas and Snowflake in a scalable, fully replayable manner
☆71Updated 3 years ago
Alternatives and similar repositories for paas-data-ingestion
Users that are interested in paas-data-ingestion are comparing it to the libraries listed below
Sorting:
- Playground for using large language models into the Modern Data Stack for entity matching☆108Updated 2 years ago
- Joining the modern data stack with the modern ML stack☆199Updated 2 years ago
- Sample project that use Dagster, dbt, DuckDB and Dash to visualize car and motorcycle Spanish market☆57Updated 2 years ago
- A guide for leading a data (engineering) team☆63Updated last year
- A write-audit-publish implementation on a data lake without the JVM☆46Updated last year
- A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.☆185Updated 2 years ago
- Supporting materials/code examples for my course in data engineering for machine learning.☆38Updated 2 years ago
- A PaaS End-to-End ML Setup with Metaflow, Serverless and SageMaker.☆37Updated 4 years ago
- Pythonic Programming Framework to orchestrate jobs in Databricks Workflow☆218Updated last month
- Recommendations at "Reasonable Scale": joining dataOps with recSys through dbt, Merlin and Metaflow☆239Updated 2 years ago
- Cost Efficient Data Pipelines with DuckDB☆57Updated 4 months ago
- Data pipeline with dbt, Airflow, Great Expectations☆163Updated 4 years ago
- Demo Project for Open Source MDS☆168Updated 2 weeks ago
- Tutorials for Fugue - A unified interface for distributed computing. Fugue executes SQL, Python, and Pandas code on Spark and Dask withou…☆114Updated last year
- Kedro Plugin to support running workflows on Kubeflow Pipelines☆55Updated 2 months ago
- A library to find and visualise the most interesting slices in multidimensional data☆109Updated 5 months ago
- Possibly the fastest DataFrame-agnostic quality check library in town.☆203Updated last week
- Sample configuration to deploy a modern data platform.☆88Updated 3 years ago
- ☆80Updated 11 months ago
- Example Dagster Cloud code for the Hooli Data Engineering organization.☆11Updated this week
- Delta Lake helper methods. No Spark dependency.☆23Updated last year
- Demo repository to lambda-fy your dbt runs☆11Updated 2 years ago
- A GitHub Action that makes it easy to use Great Expectations to validate your data pipelines in your CI workflows.☆81Updated last year
- A curated list of awesome blogs, videos, tools and resources about Data Contracts☆179Updated last year
- Demo of Streamlit application with Databricks SQL Endpoint☆34Updated 2 years ago
- 🚕 Self-contained demo using Redpanda, Materialize, River, Redis, and Streamlit to predict taxi trip durations☆45Updated 2 years ago
- A CLI tool to streamline getting started with Apache Airflow™ and managing multiple Airflow projects☆220Updated 4 months ago
- Parse dbt artifacts and search dbt models with Algolia☆52Updated 4 years ago
- A simple and easy to use Data Quality (DQ) tool built with Python.☆50Updated 2 years ago
- Get started with dbt in less than 1 minute from `git clone` to `dbt docs serve` for free!☆224Updated 4 months ago