A scalable ETL pipeline solution for seamless migration of large datasets from on-premises to Azure cloud storage. Built with Azure Data Factory, Python, SQL, and Terraform, this project optimizes both performance and cost-efficiency for enterprise data migration projects.
☆23Mar 3, 2025Updated last year
Alternatives and similar repositories for Automated-Data-Pipeline-for-Cloud-Migration
Users that are interested in Automated-Data-Pipeline-for-Cloud-Migration are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Programming solution for Hackerrank certification questions. Language: Python3☆12Aug 24, 2023Updated 3 years ago
- Real World Project on Formula1 Racing using Azure Databricks, Delta Lake and Azure Data Factory☆13Jul 24, 2023Updated 3 years ago
- Tokens → Transformers → Model Selection → Providers → Prompt Engineering → Structured Outputs → Embeddings → Vector Search → RAG → Advanc…☆18Sep 9, 2026Updated last month
- Udacity Data Analyst Degree - Project II☆11Sep 25, 2018Updated 8 years ago
- remaking Aman Kharwal's 60 Python Projects with Source Code☆17May 29, 2026Updated 5 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆21Jul 17, 2023Updated 3 years ago
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 3 years ago
- Dex lets your agent build your analytics.☆104Updated this week
- Here I will be exploring various tools and methods that are used in data engineering process with Python.☆21Jan 4, 2021Updated 5 years ago
- A GitHub repo with materials for preparing for DP-420: Microsoft Certified: Azure Cosmos DB Developer Specialty certification Exam.☆17Jul 16, 2024Updated 2 years ago
- An "End-to-End Machine Learning" project focuses on building a machine learning pipeline that prevents data leakage and deploys the model…☆18Aug 5, 2025Updated last year
- ☆25Jan 18, 2025Updated last year
- A collection of some awesome public Julia programming language projects.☆28Feb 22, 2024Updated 2 years ago
- Analytics Guide book☆38Jan 5, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- ☆40Oct 19, 2026Updated 2 weeks ago
- ☆29Jul 9, 2025Updated last year
- ☆58May 28, 2026Updated 5 months ago
- ☆32Sep 10, 2019Updated 7 years ago
- A selection of small Data Science Projects.☆32Dec 23, 2022Updated 3 years ago
- Project - Data Processing and Analysis in Python Course☆39Oct 10, 2018Updated 8 years ago
- Cost Efficient Data Pipelines with DuckDB☆63May 14, 2025Updated last year
- Case study solutions for #8WeekSQLChallenge by Danny Ma.☆45Jan 9, 2023Updated 3 years ago
- In this repository, I will try to build a machine learning model that can predict the return in Gold over short term windows using histor…☆44Apr 30, 2020Updated 6 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)☆41Aug 9, 2024Updated 2 years ago
- 100 essential Databricks concepts for data engineers, organized by category with difficulty levels and self-assessment scoring☆57May 28, 2026Updated 5 months ago
- 170+ curated resources every Databricks Data Engineer should bookmark - tools, courses, creators, labs, and communities☆59Oct 22, 2026Updated last week
- This repository focuses on providing interview scenario questions that I have encountered during interviews. The questions are designed t…☆66Oct 16, 2026Updated 2 weeks ago
- Transformer based Bangla Speech Recognition | Encoder Decoder Architecture☆64Apr 14, 2023Updated 3 years ago
- Azure Data Engineering Cookbook, published by Packt☆61Jan 18, 2023Updated 3 years ago
- Julia Projects for Beginners — Easy Ideas to Get Started Coding in Julia☆78Feb 22, 2022Updated 4 years ago
- Cracking Data Engineering Interview Guide, published by Packt☆63Nov 10, 2023Updated 2 years ago
- ☆59Jul 9, 2023Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- #8WeekSQLChallenge by Danny Ma.☆90Apr 8, 2022Updated 4 years ago
- Collection of LeetCode SQL questions to ace the coding interview!!☆81Jul 10, 2023Updated 3 years ago
- scikit-learn Cookbook Second Edition, published by Packt☆111Jan 30, 2023Updated 3 years ago
- Contains solved queries for the Hackerrank SQL (Basic) Skills Certification Test 🎓.☆107Aug 24, 2022Updated 4 years ago
- Code Repository for The Kaggle Book 2nd Edition, Published by Packt☆122Mar 2, 2026Updated 8 months ago
- A Model Context Protocol (MCP) server that enables AI assistants to securely access and analyze Microsoft Fabric Analytics data through a…☆110Dec 23, 2025Updated 10 months ago
- YouTube tutorial project☆112Oct 17, 2023Updated 3 years ago