KennethanCeyer / awesome-data-pipeline
Awesome list for datapipeline
☆28Updated last year
Related projects ⓘ
Alternatives and complementary repositories for awesome-data-pipeline
- New generation opensource data stack☆61Updated 2 years ago
- 📒(GitBook) A curated list of awesome Data Engineering resources☆34Updated 3 weeks ago
- A curated list of awesome blogs, videos, tools and resources about Data Contracts☆166Updated 3 months ago
- Code snippets for Data Engineering Design Patterns book☆40Updated last week
- A curated list of awesome DataOps tools☆158Updated last month
- Spark data pipeline that processes movie ratings data.☆27Updated last week
- Design/Implement stream/batch architecture on NYC taxi data | #DE☆26Updated 3 years ago
- ☆15Updated 3 months ago
- Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)☆44Updated last year
- Quick Guides from Dremio on Several topics☆65Updated 3 weeks ago
- Jupyter notebooks and AWS CloudFormation template to show how Hudi, Iceberg, and Delta Lake work☆47Updated 2 years ago
- Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pi…☆92Updated 3 weeks ago
- Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observ…☆111Updated last week
- A Snowflake GPT Demo using SqlAlchemy☆23Updated last year
- A curated list of dagster code snippets for data engineers☆52Updated 8 months ago
- Docker Airflow - Contains a docker compose file for Airflow 2.0☆59Updated 2 years ago
- Full stack data engineering tools and infrastructure set-up☆44Updated 3 years ago
- Spark Structured Streaming data pipeline that processes movie ratings data in real-time.☆13Updated last week
- A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in …☆21Updated 2 years ago
- Support for generating modern platforms dynamically with services such as Kafka, Spark, Streamsets, HDFS, ....☆72Updated this week
- A repository of sample code to show data quality checking best practices using Airflow.☆72Updated last year
- ☆32Updated 8 months ago
- Curated list of resources about Apache Airflow☆19Updated 3 years ago
- A curated list of awesome Databricks resources, including Spark☆14Updated 4 months ago
- Data Tools Subjective List☆80Updated last year
- Sample configuration to deploy a modern data platform.☆86Updated 2 years ago
- Make dbt great again! Enables end user to extend dbt to his/her needs☆13Updated this week
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆71Updated last year
- This is a Vs Code extension for Apache Airflow☆20Updated 11 months ago
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆62Updated last month