DeepHiveMind / Modern_E2E_ServerlessDataPipeline_NextGenDataLake
Next Gen Serverless E2E Data Pipeline & Workflow Orchestration | Modern Data Lake |
☆10Updated 4 years ago
Alternatives and similar repositories for Modern_E2E_ServerlessDataPipeline_NextGenDataLake:
Users that are interested in Modern_E2E_ServerlessDataPipeline_NextGenDataLake are comparing it to the libraries listed below
- Blockchain framework/service: Hyperledger Fabric, AWS-QLDB & AWS-MSB☆13Updated 4 years ago
- Real-World AI/ML Ecosystem | Enterprise AI Platform Recipe | Custom MLOPS☆20Updated 2 years ago
- This project is about building a dimensional data warehouse in BigQuery by transforming an OLTP system to an OLAP system, using dbt as ou…☆10Updated last year
- ETL pipeline using pyspark (Spark - Python)☆112Updated 4 years ago
- Companion repository for the book 'Delta Lake Up and Running'☆45Updated 8 months ago
- PySpark functions and utilities with examples. Assists ETL process of data modeling☆99Updated 4 years ago
- ☆45Updated last year
- GCP-Data-Engineer-Study-Guide☆118Updated 5 years ago
- Docker with Airflow and Spark standalone cluster☆247Updated last year
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- [DEPRECATED] Demo repository implementing an end-to-end MLOps workflow on Databricks. Project derived from dbx basic python template☆109Updated last year
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆166Updated last year
- PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster☆438Updated 3 months ago
- Local Environment to Practice Data Engineering☆120Updated 2 weeks ago
- A tutorial for the Great Expectations library.☆69Updated 3 years ago
- Databricks - Apache Spark™ - 2X Certified Developer☆265Updated 4 years ago
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆135Updated 4 years ago
- ☆44Updated last year
- ☆23Updated last year
- Spark data pipeline that processes movie ratings data.☆27Updated this week
- Guide for databricks spark certification☆58Updated 3 years ago