KennethanCeyer / awesome-data-pipelineLinks
Awesome list for datapipeline
☆35Updated 2 years ago
Alternatives and similar repositories for awesome-data-pipeline
Users that are interested in awesome-data-pipeline are comparing it to the libraries listed below
Sorting:
- A curated list of awesome DataOps tools☆215Updated 2 weeks ago
- ☆76Updated last week
- Full stack data engineering tools and infrastructure set-up☆57Updated 4 years ago
- Awesome list of dataops products, open source and resources☆24Updated 3 years ago
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆91Updated 2 years ago
- Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)☆64Updated 2 years ago
- A curated list of awesome open source tools and commercial products to catalog, version, and manage data 🚀☆39Updated 3 years ago
- A curated list of awesome blogs, videos, tools and resources about Data Contracts☆180Updated last year
- Repo for everything open table formats (Iceberg, Hudi, Delta Lake) and the overall Lakehouse architecture☆125Updated last month
- A curated list of open source tools used in analytics platforms and data engineering ecosystem☆412Updated 9 months ago
- A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for …☆141Updated 5 years ago
- 📙 Awesome Data Catalogs and Observability Platforms.☆955Updated 4 months ago
- New generation opensource data stack☆76Updated 3 years ago
- Installer for DataKitchen's Open Source Data Observability Products. Data breaks. Servers break. Your toolchain breaks. Ensure your team …☆129Updated last month
- A real-time event pipeline around Kafka Ecosystem for Chicago Transit Authority.☆32Updated 2 years ago
- Auto-generated Diagrams from Airflow DAGs. 🔮 🪄☆354Updated this week
- Data Tools Subjective List☆88Updated 2 years ago
- Project files for the post: Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for A…☆41Updated 3 years ago
- Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python☆44Updated 2 years ago
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆44Updated last year
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆28Updated 3 years ago
- Spark data pipeline that processes movie ratings data.☆30Updated last week
- Build Data Lake using Open Source tools☆117Updated 7 months ago
- Test data management tool for any data source, batch or real-time. Generate, validate and clean up data all in one tool.☆72Updated last week
- A curated list of resources about Snowflake☆253Updated last year
- dlt-dagster-demo☆13Updated 2 years ago
- A curated list of awesome Databricks resources, including Spark☆22Updated last year
- Apache Airflow advanced functionalities examples☆21Updated last year
- How to build an awesome data engineering team☆101Updated 6 years ago
- Data pipeline that scrapes Rust cheater Steam profiles☆54Updated 3 years ago