Transform data from on-premises SQL Server to Azure Delta Lake Storage for Analytics and Visualization
☆21Jul 16, 2023Updated 2 years ago
Alternatives and similar repositories for Azure-Data-Engineering
Users that are interested in Azure-Data-Engineering are comparing it to the libraries listed below
Sorting:
- End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)☆28Aug 9, 2024Updated last year
- This repository contains SQL queries from various popular online learning resources e.g. Vertabelo Academy, SQLZoo etc.☆44Jun 22, 2019Updated 6 years ago
- In this repository, you are going to learn the fundamentals of SQL: how to create a SQL database from scratch, how to use SQL commands, h…☆74Jun 30, 2024Updated last year
- ☆13Feb 14, 2025Updated last year
- ☆14May 14, 2024Updated last year
- ☆13Apr 18, 2024Updated last year
- This Repo contains Jupyter Notebooks to recap on RDD, DataFrame, Spark Streaming and ML operations using Pyspark☆11Nov 3, 2024Updated last year
- Using a combination of Excel, SQL, and Tableau, I delved into the extensive datasets comprising over 82k rows of data from Netflix's show…☆102Jul 10, 2023Updated 2 years ago
- This project aims to move the data from a Relational database system (RDBMS) to a Hadoop file system (HDFS)☆11Apr 29, 2022Updated 3 years ago
- Data Engineering pipeline hosted entirely in the AWS ecosystem utilizing DocumentDB as the database☆14Oct 26, 2021Updated 4 years ago
- A data pipeline for processing football data using Python and SQL☆13Sep 12, 2023Updated 2 years ago
- ☆17Feb 1, 2025Updated last year
- Python - Complete Python, Django, Data Science and ML Guide, published by Packt☆15Dec 15, 2025Updated 3 months ago
- A project portfolio to accompany my resume☆30Sep 5, 2023Updated 2 years ago
- ☆31Nov 14, 2024Updated last year
- ☆29Oct 24, 2024Updated last year
- ☆32Mar 7, 2018Updated 8 years ago
- Fully automated csv to dashboard pipeline using Terraform, Google Cloud Storage, BigQuery, dbt, Prefect and Looker Studio. Peer ranked …☆41Nov 15, 2024Updated last year
- ☆28Jul 9, 2025Updated 8 months ago
- Repo containing all of my Data engineering projects☆13May 4, 2025Updated 10 months ago
- ☆12Jul 22, 2025Updated 7 months ago
- Use of computer vision to detect shoplifting activity and report it into a smart dashboard☆18Oct 16, 2024Updated last year
- ☆19Jun 22, 2022Updated 3 years ago
- A course by DataTalks Club that covers Spark, Kafka, Docker, Airflow, Terraform, DBT, Big Query etc☆16Mar 18, 2022Updated 4 years ago
- For the Coursera specialization https://www.coursera.org/specializations/gcp-data-machine-learning☆100Nov 22, 2017Updated 8 years ago
- An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Az…☆32Oct 2, 2023Updated 2 years ago
- Outputs the sentiment for the top 100 subreddits. Also included is a scraper for writing an updated top 100 subreddits to a text file.☆22Mar 18, 2020Updated 6 years ago
- This project builds an End-to-End Azure Data Engineering Pipeline, performing ETL and Analytics Reporting on the AdventureWorks2017LT Dat…☆20Feb 19, 2025Updated last year
- Machine Learning Course @ Santa Clara University☆24Jun 10, 2020Updated 5 years ago
- Databricks DLT Apparel Pipeline Project: Learn medallion architecture, streaming, and data engineering with Delta Live Tables. Includes s…☆40Feb 25, 2026Updated 3 weeks ago
- Data Engineering Project: Extracting music video metrics of Twice using YouTube API, AWS, and Tableau☆32Nov 21, 2023Updated 2 years ago
- Personal project for setting up an open source data warehouse.☆32Jul 11, 2025Updated 8 months ago
- Airflow Tutorials☆25Feb 28, 2021Updated 5 years ago
- ☆40Feb 2, 2025Updated last year
- Feature demos, integration guides & hands-on labs/projects using Kpow, Flex, Kafka, Flink, Iceberg & more☆50Mar 13, 2026Updated last week
- ☆28Jul 20, 2025Updated 8 months ago
- Data Engineering Project with Hadoop HDFS and Kafka☆123Nov 4, 2023Updated 2 years ago
- Data Structures and Algorithms (DSA) is a fundamental part of Computer Science that teaches you how to think and solve complex problems s…☆28Jun 20, 2025Updated 9 months ago
- In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…☆25May 6, 2023Updated 2 years ago