jw-ng / airflow-dynamic-dags

Related projects ⓘ

Alternatives and complementary repositories for airflow-dynamic-dags

• hnawaz007 / dbt-dw
  build dw with dbt
  ☆29Updated 3 weeks ago
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆44Updated 2 years ago
• 1997mahadi / dbt-dlt-ingestion-pipeline
  ☆15Updated 3 months ago
• ongxuanhong / de03-trino-dbt-spark-everything-everywhere-all-at-once
  ☆15Updated 9 months ago
• simardeep1792 / Data-Engineering-Streaming-Project
  ☆38Updated 4 months ago
• delta-io / delta-docs
  Delta Lake Documentation
  ☆46Updated 5 months ago
• astronomer / airflow-data-quality-demo
  A repository of sample code to show data quality checking best practices using Airflow.
  ☆72Updated last year
• ankurchavda / data-engineering-zoomcamp
  A course by DataTalks Club that covers Spark, Kafka, Docker, Airflow, Terraform, DBT, Big Query etc
  ☆11Updated 2 years ago
• ssp-data / data-engineering-devops
  Full stack data engineering tools and infrastructure set-up
  ☆44Updated 3 years ago
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• delta-io / delta-examples
  Delta Lake examples
  ☆208Updated last month
• dbt-labs / dbt-starter-project
  Cloned by the `dbt init` task
  ☆59Updated 6 months ago
• josephmachado / online_store
  End to end data engineering project
  ☆51Updated 2 years ago
• astronomer / airflow-covid-data
  Sample Airflow DAGs to load data from the CovidTracking API to Snowflake via an AWS S3 intermediary.
  ☆16Updated 3 years ago
• ddgope / Data-Pipelines-with-Airflow
  This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging…
  ☆74Updated 5 years ago
• karenbajador / pyspark_greatexpectations
  ☆11Updated 2 years ago
• Stefen-Taime / Iceberg-Dbt-Trino-Hive-modern-open-source-data-stack
  To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…
  ☆28Updated 8 months ago
• omidvd79 / Big_Data_Demystified
  Big Data Demystified meetup and blog examples
  ☆31Updated 3 months ago
• danielbeach / unitTestPySpark
  how to unit test your PySpark code
  ☆28Updated 3 years ago
• ismaildawoodjee / GreatEx
  A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in …
  ☆21Updated 2 years ago
• jamesshocking / Spark-REST-API-UDF
  Example of how to leverage Apache Spark distributed capabilities to call REST-API using a UDF
  ☆50Updated 2 years ago
• bartosz25 / data-engineering-design-patterns-book
  Code snippets for Data Engineering Design Patterns book
  ☆40Updated last week
• Dorianteffo / modern-data-platform
  End-to-end data platform leveraging the Modern data stack
  ☆41Updated 7 months ago
• gmyrianthous / dbt-airflow
  A Python package that creates fine-grained dbt tasks on Apache Airflow
  ☆62Updated last month
• franloza / coches-net-dashboard
  Sample project that use Dagster, dbt, DuckDB and Dash to visualize car and motorcycle Spanish market
  ☆55Updated last year
• dominikhei / Local-Data-LakeHouse
  Sample Data Lakehouse deployed in Docker containers using Apache Iceberg, Minio, Trino and a Hive Metastore. Can be used for local testin…
  ☆57Updated last year
• astronomer / airflow-testing-guide
  ☆20Updated 3 years ago
• sdw-online / postgres-dwh
  A Postgres data warehouse for processing synthetic data using IAC principles
  ☆16Updated last year
• josephmachado / simple_dbt_project
  Code for dbt tutorial
  ☆143Updated 5 months ago