anilkulkarni87 / airflow-docker

Related projects ⓘ

Alternatives and complementary repositories for airflow-docker

• shravan-kuchkula / udacity-data-eng-proj3
  Built a stream processing data pipeline to get data from disparate systems into a dashboard using Kafka as an intermediary.
  ☆28Updated last year
• yennanliu / NYC_Taxi_Pipeline
  Design/Implement stream/batch architecture on NYC taxi data | #DE
  ☆26Updated 3 years ago
• karenbajador / pyspark_greatexpectations
  ☆11Updated 2 years ago
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆44Updated 2 years ago
• feluelle / finance-data-builder
  Finance 🏦 Data Builder 🛠️ @ postgres 🐘
  ☆18Updated 3 years ago
• vim89 / datapipelines-essentials-python
  Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…
  ☆53Updated last year
• Wittline / pyspark-on-aws-emr
  The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on…
  ☆26Updated 2 years ago
• hnawaz007 / dbt-dw
  build dw with dbt
  ☆29Updated last month
• TJaniF / airflow-elt-blueprint
  A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.
  ☆58Updated last year
• rodrigo-arenas / kafkaml-anomaly-detection
  Project for real-time anomaly detection using Kafka and python
  ☆56Updated last year
• sankamuk / PysparkCheatsheet
  PySpark Cheatsheet
  ☆35Updated last year
• datastacktv / apache-beam-batch-processing
  Public source code for the Batch Processing with Apache Beam (Python) online course
  ☆19Updated 4 years ago
• uplimit / course-dbt
  Analytics engineering with dbt - projects and developer environment
  ☆16Updated last month
• josephmachado / beginner_de_project_stream
  Simple stream processing pipeline
  ☆92Updated 5 months ago
• ssp-data / data-engineering-devops
  Full stack data engineering tools and infrastructure set-up
  ☆44Updated 3 years ago
• Pathairush / rdbms_to_hdfs_data_pipeline
  A data pipeline moving data from a Relational database system (RDBMS) to a Hadoop file system (HDFS).
  ☆15Updated 3 years ago
• jmeisele / ml-ops-kafka
  Kafka variant of the MLOps Level 1 stack
  ☆21Updated 2 years ago
• delta-io / delta-docs
  Delta Lake Documentation
  ☆47Updated 5 months ago
• josephmachado / e2e_datapipeline_test
  Example repo to create end to end tests for data pipeline.
  ☆21Updated 5 months ago
• cordon-thiago / spark-schema-merge
  Spark app to merge different schemas
  ☆23Updated 3 years ago
• simardeep1792 / Data-Engineering-Streaming-Project
  ☆38Updated 4 months ago
• bartosz25 / data-engineering-design-patterns-book
  Code snippets for Data Engineering Design Patterns book
  ☆40Updated this week
• Dorianteffo / modern-data-platform
  End-to-end data platform leveraging the Modern data stack
  ☆41Updated 7 months ago
• soumilshah1995 / Airflow-Tutorials-
  Airflow Tutorials
  ☆24Updated 3 years ago
• Aiven-Labs / python-fake-data-producer-for-apache-kafka
  The Python fake data producer for Apache Kafka® is a complete demo app allowing you to quickly produce JSON fake streaming datasets and …
  ☆83Updated 6 months ago
• jw-ng / airflow-dynamic-dags
  ☆13Updated 2 years ago
• sdw-online / postgres-dwh
  A Postgres data warehouse for processing synthetic data using IAC principles
  ☆16Updated last year
• yennanliu / DE-100-days
  data engineering 100 days 🤖 🧲 🦾 | #DE
  ☆37Updated last year
• cnstlungu / portable-data-stack-airflow
  A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset
  ☆27Updated 2 weeks ago