reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.
☆15Jun 26, 2023Updated 3 years ago
Alternatives and similar repositories for modern-data-pipeline
Users that are interested in modern-data-pipeline are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Data Pipeline from the Global Historical Climatology Network DataSet☆27Dec 20, 2022Updated 3 years ago
- In this article, you will learn how to set up a real-time data processing and analytics environment using Docker, MySQL, Redpanda, MinIO,…☆11Jun 27, 2023Updated 3 years ago
- The "One Role to Rule them all" Ansible setup☆13Sep 20, 2019Updated 7 years ago
- Example repo to create end to end tests for data pipeline.☆25Jun 14, 2024Updated 2 years ago
- a lightweight, semi-automated setup guide for HashiStack: Consul + Vault + Nomad, on Footloose powered Docker "container VMs", with Ansib…☆11Jul 2, 2021Updated 5 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- This portfolio is tiny, it saves the environment 😮☆14May 8, 2026Updated 6 months ago
- In this project I used apache airflow to scrape website periodically. This is for the tutorials I do on youtube.☆10Nov 21, 2022Updated 3 years ago
- StatusNotifier for xfce4-panel and vala-panel - GitHub mirror☆11Apr 12, 2019Updated 7 years ago
- ☆16Jul 15, 2023Updated 3 years ago
- Ansible setup of my development workstation☆13Sep 22, 2023Updated 3 years ago
- Tips on Kubernetes cluster management using kubectl command☆10Dec 28, 2022Updated 3 years ago
- A xfce4-panel plugin that shows activate window title and icon (quite useful in some tiling window managers)☆18Apr 9, 2021Updated 5 years ago
- Resources and projects from Udacity Data Engineering with AWS nano degree programme☆30Apr 12, 2023Updated 3 years ago
- Deploy google adk on bare metal☆12Sep 21, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Garage (distributed, s3 compatible object storage service) setup using Ansible and Docker☆19Mar 15, 2023Updated 3 years ago
- Building Event Driven Application with AWS Lambda and Amazon Redshift Data API☆17Oct 27, 2020Updated 6 years ago
- A Proof of concept of a modern python CLI with click, pydantic, rich and anyio☆21Jan 13, 2024Updated 2 years ago
- my personal portfolio☆13Sep 6, 2025Updated last year
- I am using confluent Kafka cluster to produce and consume scraped data. In this project, I've created a real-time data pipeline that uti…☆29May 2, 2023Updated 3 years ago
- In this end to end project I have built a RAG app using ObjectBox Vector Databse and LangChain. With Objectbox you can do OnDevice AI, wi…☆12May 26, 2024Updated 2 years ago
- This is a public repository that the dbt proserv team uses for collective demos.☆16Mar 20, 2026Updated 7 months ago
- Data pipeline for EIA datasets.☆12Apr 8, 2026Updated 7 months ago
- A pretty resume template in Typst | 一个美观的typst简历模板☆17Feb 12, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- AlvinToh Learning Repository for The Ultimate Hands-On Hadoop - Tame your Big Data!☆10May 23, 2018Updated 8 years ago
- Scalable CDC Pattern Implemented using PySpark☆18Oct 8, 2025Updated last year
- ☆30Nov 16, 2023Updated 2 years ago
- A project portfolio to accompany my resume☆32Sep 5, 2023Updated 3 years ago
- End to end data pipeline to extract and analyze submissions from any subreddit using Pushshift, python, dbt and BigQuery.☆13Jul 17, 2023Updated 3 years ago
- hackintosh 13900 macos☆12Jun 23, 2023Updated 3 years ago
- Javascript for rendering CSS animations of Chinese character stroke data☆14Dec 14, 2015Updated 10 years ago
- ☆29Jan 12, 2023Updated 3 years ago
- example how to perform distributed bayesian optimisation (autoML) using optuna on metaflow☆10Oct 6, 2021Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- My typst templates & utilities (WIP)☆27Oct 21, 2025Updated last year
- Examples which shows how to create MATE panel applet with Python.☆18Sep 14, 2019Updated 7 years ago
- Maps Medicare LDS claims data to the Tuva Input Layer so you can easily run the Tuva Project.☆13Dec 15, 2025Updated 10 months ago
- RustBuy is a simple command-line interface (CLI) e-commerce application built in Rust, designed specifically to help you understand and p…☆20Sep 26, 2025Updated last year
- Final Project for Data Engineering Zoomcamp Course 2024 🧙🔥☆11Apr 17, 2024Updated 2 years ago
- Python script that utilizes the Datto API to pull data about device and asset errors.☆11Jan 10, 2022Updated 4 years ago
- ☆12Oct 15, 2023Updated 3 years ago
Are these results useful?