TJaniF / airflow-kafka-quickstart

Related projects ⓘ

Alternatives and complementary repositories for airflow-kafka-quickstart

• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆44Updated 2 years ago
• josephmachado / online_store
  End to end data engineering project
  ☆49Updated 2 years ago
• marclamberti / webinar-airflow-chart
  Materials of the Official Helm Chart Webinar
  ☆27Updated 3 years ago
• bartosz25 / data-engineering-design-patterns-book
  Code snippets for Data Engineering Design Patterns book
  ☆38Updated this week
• yTek01 / docker-spark-airflow
  ☆31Updated last year
• josephmachado / socialetl
  Project for "Data pipeline design patterns" blog.
  ☆41Updated 3 months ago
• pyjaime / docker-airflow-spark
  Docker with Airflow + Postgres + Spark cluster + JDK (spark-submit support) + Jupyter Notebooks
  ☆21Updated 2 years ago
• astronomer / airflow-data-quality-demo
  A repository of sample code to show data quality checking best practices using Airflow.
  ☆72Updated last year
• airscholar / changecapture-e2e
  This project shows how to capture changes from postgres database and stream them into kafka
  ☆31Updated 5 months ago
• josephmachado / e2e_datapipeline_test
  Example repo to create end to end tests for data pipeline.
  ☆21Updated 4 months ago
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• marcosmarxm / airflow-testing-ci-workflow
  (project & tutorial) dag pipeline tests + ci/cd setup
  ☆85Updated 3 years ago
• itversity / data-engineering-spark
  ☆86Updated 2 years ago
• hnawaz007 / dbt-dw
  build dw with dbt
  ☆29Updated 2 weeks ago
• dsynkov / spark-livy-on-airflow-workspace
  A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.
  ☆39Updated 3 years ago
• josephmachado / docker_for_data_engineers
  Code for blog at: https://www.startdataengineering.com/post/docker-for-de/
  ☆30Updated 6 months ago
• josephmachado / analytical_dp_with_sql
  Code for my "Efficient Data Processing in SQL" book.
  ☆49Updated 3 months ago
• greatexpectationslabs / ge_tutorials
  Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.
  ☆166Updated last year
• TJaniF / airflow-elt-blueprint
  A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.
  ☆58Updated last year
• tobigiwa / data-engineering-book-reviews
  Just starting your DE journey or along the way already?. I will be sharing a short list of DATA-ENGINEERING-CENTRED books that covers the…
  ☆34Updated 2 years ago
• josephmachado / data_engineering_best_practices
  Sample project to demonstrate data engineering best practices
  ☆164Updated 8 months ago
• Snowflake-Labs / sfguide-data-engineering-with-snowpark-python
  ☆111Updated last month
• Data-Engineering-Weekly / dataengineeringweekly
  Weekly Data Engineering Newsletter
  ☆93Updated 3 months ago
• 1ambda / lakehouse
  Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)
  ☆43Updated last year
• akashmehta10 / cdc_pyspark_hive
  ☆22Updated last year
• simardeep1792 / Data-Engineering-Streaming-Project
  ☆37Updated 4 months ago
• cordon-thiago / airflow-spark
  Docker with Airflow and Spark standalone cluster
  ☆244Updated last year
• delta-io / delta-docs
  Delta Lake Documentation
  ☆46Updated 4 months ago
• developershomes / SparkETL
  Spark all the ETL Pipelines
  ☆32Updated last year