Wittline / pyDag
Scheduling Big Data Workloads and Data Pipelines in the Cloud with pyDag
☆24Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for pyDag
- dagster scikit-learn pipeline example.☆43Updated last year
- Challenge Data Engineer☆25Updated 2 years ago
- A template DBT project for BigQuery on Google Cloud☆12Updated 3 years ago
- The go to demo for public and private dbt Learn☆69Updated 2 months ago
- ☆11Updated 2 years ago
- Cost Efficient Data Pipelines with DuckDB☆46Updated 3 months ago
- Getting Great Expectations setup to run on DataBricks with Spark Dataframes.☆12Updated 2 years ago
- ☆22Updated 2 years ago
- A proof of concept for how to set up a codebase for an analytics org.☆14Updated 3 years ago
- Data lake, data warehouse on GCP☆54Updated 2 years ago
- Build your feature store with macros right within your dbt repository☆37Updated last year
- The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on…☆26Updated 2 years ago
- dbt Cloud pipelines in airflow examples☆35Updated last year
- Snowflake Cookbook, published by Packt☆73Updated last year
- ☆15Updated 3 months ago
- dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.☆56Updated 2 years ago
- Debussy is an opinionated Data Architecture and Engineering framework, enabling data analysts and engineers to build better platforms and…☆28Updated last year
- Code for my "Efficient Data Processing in SQL" book.☆50Updated 3 months ago
- Spark Application UI extension for JupyterLab☆10Updated 3 years ago
- This repository contains an example of how to leverage Cloud Composer and Cloud Dataflow to move data from a Microsoft SQL Server to BigQ…☆17Updated 6 months ago
- Repo demonstrating a Dagster pipeline to generate Neo4j Graph☆21Updated 3 years ago
- A _simple_ starter template for Snowflake Cloud Data Platform☆39Updated 2 years ago
- Execution of DBT models using Apache Airflow through Docker Compose☆113Updated last year
- DuckDB SQL Tools add DuckDB support to VSCode, and provide database schema and SQL query interfaces for the popular SQLTools extension, S…☆12Updated 4 months ago
- Sample project that use Dagster, dbt, DuckDB and Dash to visualize car and motorcycle Spanish market☆55Updated last year
- Full stack data engineering tools and infrastructure set-up☆44Updated 3 years ago
- Creates simple data models on Snowflake to report dbt source freshness and tests☆23Updated last year
- A simple Data Engineering solution for testing or education purposes. You only need to know SQL and Python to understand this project. Da…☆24Updated 2 years ago
- Getting started with DuckDB, by Packt Publishing☆43Updated 3 months ago