Alternatives and similar repositories for bigdata-ETL-pipeline:

Users that are interested in bigdata-ETL-pipeline are comparing it to the libraries listed below

• harrydevforlife / building-lakehouse
  Building Data Lakehouse by open source technology. Support end to end data pipeline, from source data on AWS S3 to Lakehouse, visualize a…
  ☆17Updated 9 months ago
• sdw-online / postgres-dwh
  A Postgres data warehouse for processing synthetic data using IAC principles
  ☆15Updated last year
• hnawaz007 / dbt-dw
  build dw with dbt
  ☆33Updated 2 months ago
• cnstlungu / portable-data-stack-airflow
  A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset
  ☆33Updated 2 months ago
• 1997mahadi / dbt-dlt-ingestion-pipeline
  ☆18Updated 5 months ago
• arezamoosavi / AcidOnSpark-ETL
  Delta-Lake, ETL, Spark, Airflow
  ☆45Updated 2 years ago
• ssp-data / data-engineering-devops
  Full stack data engineering tools and infrastructure set-up
  ☆47Updated 3 years ago
• josephmachado / cost_effective_data_pipelines
  Cost Efficient Data Pipelines with DuckDB
  ☆48Updated 5 months ago
• anilkulkarni87 / airflow-docker
  This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and …
  ☆27Updated 11 months ago
• danielbeach / tinytimmy
  A simple and easy to use Data Quality (DQ) tool built with Python.
  ☆49Updated last year
• alonsomedo / os-data-stack
  Building a Data Pipeline with an Open Source Stack
  ☆41Updated 6 months ago
• karenbajador / pyspark_greatexpectations
  ☆11Updated 2 years ago
• mrjsj / delta-lake-explorer
  ☆26Updated 6 months ago
• josephmachado / analytical_dp_with_sql
  Code for my "Efficient Data Processing in SQL" book.
  ☆54Updated 5 months ago
• khuyentran1401 / prefect-docker
  Demo on how to use Prefect with Docker
  ☆25Updated 2 years ago
• dqops / dqo
  Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observ…
  ☆124Updated last week
• jeantardelli / data-engineering-with-python
  Here I will be exploring various tools and methods that are used in data engineering process with Python.
  ☆22Updated 4 years ago
• Dorianteffo / modern-data-platform
  End-to-end data platform leveraging the Modern data stack
  ☆43Updated 9 months ago
• airscholar / modern-data-eng-dbt-databricks-azure
  In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …
  ☆24Updated last year
• gmyrianthous / dbt-airflow
  A Python package that creates fine-grained dbt tasks on Apache Airflow
  ☆62Updated 3 months ago
• RyanEricLamb / data-engineering-bus-tracker
  Data engineering project using UK Bus Open Data Service (BODS) to calculate late buses in real-time for any selected region in England. P…
  ☆28Updated last year
• dbt-labs / dbt-starter-project
  Cloned by the `dbt init` task
  ☆60Updated 8 months ago
• KentHsu / Udacity-Data-Streaming-Nanodegree
  Udacity Data Streaming Nanodegree Program
  ☆22Updated 3 years ago
• bgarcevic / danish-democracy-data
  ☆31Updated 3 weeks ago
• mito-ds / snowflake-streamlit-bank-performance-demo
  A demo of the Mito Streamlit Spreadsheet
  ☆17Updated last year
• zsvoboda / ngods
  New generation opensource data stack
  ☆65Updated 2 years ago
• airbytehq / open-data-stack
  Open Data Stack Projects: Examples of End to End Data Engineering Projects
  ☆70Updated last year
• Stefen-Taime / Iceberg-Dbt-Trino-Hive-modern-open-source-data-stack
  To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…
  ☆30Updated 10 months ago
• josephmachado / data-quality-w-greatexpectations
  Code for data quality with greatexpectations blog
  ☆12Updated 5 months ago
• simardeep1792 / Data-Engineering-Streaming-Project
  ☆40Updated 6 months ago