KennethanCeyer / awesome-data-pipeline
Awesome list for datapipeline
☆31Updated last year
Alternatives and similar repositories for awesome-data-pipeline:
Users that are interested in awesome-data-pipeline are comparing it to the libraries listed below
- 📒(GitBook) A curated list of awesome Data Engineering resources☆34Updated 2 weeks ago
- Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)☆49Updated last year
- Apache Spark Guide☆30Updated 2 years ago
- Full stack data engineering tools and infrastructure set-up☆48Updated 3 years ago
- A curated list of awesome blogs, videos, tools and resources about Data Contracts☆171Updated 5 months ago
- Spark data pipeline that processes movie ratings data.☆27Updated last week
- A real-time event pipeline around Kafka Ecosystem for Chicago Transit Authority.☆29Updated last year
- Data lake, data warehouse on GCP☆55Updated 3 years ago
- ☆11Updated 2 years ago
- A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for …☆133Updated 4 years ago
- System Design, Solution Architecture, Data Systems Practice☆35Updated last month
- Apache Airflow advanced functionalities examples☆14Updated 10 months ago
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆73Updated last year
- A curated list of open source tools used in analytics platforms and data engineering ecosystem☆195Updated 2 months ago
- A curated list of awesome DataOps tools☆169Updated 3 months ago
- Code snippets for Data Engineering Design Patterns book☆53Updated 3 weeks ago
- Jupyter notebooks and AWS CloudFormation template to show how Hudi, Iceberg, and Delta Lake work☆48Updated 2 years ago
- Quick Guides from Dremio on Several topics☆67Updated 2 weeks ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆26Updated 2 years ago
- A guide for leading a data (engineering) team☆62Updated 8 months ago
- End-to-end data platform leveraging the Modern data stack☆45Updated 9 months ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- New generation opensource data stack☆65Updated 2 years ago
- A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset☆38Updated 2 months ago
- Public source code for the Batch Processing with Apache Beam (Python) online course☆18Updated 4 years ago
- build dw with dbt☆35Updated 3 months ago
- Code for dbt tutorial☆149Updated 8 months ago
- A curated list of awesome Databricks resources, including Spark☆16Updated 7 months ago
- For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR☆65Updated 3 years ago
- Cloned by the `dbt init` task☆60Updated 9 months ago