abeltavares/batch-data-pipeline external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

abeltavares/batch-data-pipeline

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/abeltavares/batch-data-pipeline)

Close

abeltavares / batch-data-pipelineView on GitHubMore

• External links
• Readme badge

🦆 Batch data pipeline with Airflow, DuckDB, Delta Lake, Trino, MinIO, and Metabase. Full observability and data quality.

☆90Nov 5, 2025Updated 8 months ago

Alternatives and similar repositories for batch-data-pipeline

Users that are interested in batch-data-pipeline are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• EnergyWeatherAI / SolarSTEPS

  View on GitHub
  ☆13Jan 15, 2026Updated 5 months ago
• udacity / nd0821-c3-deploying-a-scalable-pipeline-in-prod-demos

  View on GitHub
  Demo files for the Course - Deploying a Scalable Pipeline in Production
  ☆12Apr 15, 2026Updated 2 months ago
• zsylvester / curvaturepy

  View on GitHub
  Code for analyzing the relationship between curvature and migration rate in meandering rivers
  ☆14Dec 8, 2022Updated 3 years ago
• tcmlabs / hexagonal-architecture-python-spark

  View on GitHub
  Hexagonal (ports and adapters) architecture applied to Spark and Python data engineering project
  ☆33Jul 26, 2023Updated 2 years ago
• lucianonooijen / dotfiles

  View on GitHub
  My dotfiles used on MacOS (Arch Linux on `linux` branch). Include custom scripts and configs for NeoVim, ZSH, Tmux, Ranger, and more.
  ☆10Feb 9, 2026Updated 5 months ago
• End-to-end encrypted email - Proton Mail • Ad
  Special offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
• pedropark99 / Introd-pyspark

  View on GitHub
  An open and introductory book for the Python API of Apache Spark (pyspark) 📚📖
  ☆12Sep 19, 2025Updated 9 months ago
• lvorspel / DiGriPy

  View on GitHub
  District Heat Grid Simulation in Python
  ☆16Mar 18, 2022Updated 4 years ago
• JChz6 / DE-ZCamp-Project

  View on GitHub
  ☆15Mar 29, 2024Updated 2 years ago
• aws-samples / amazon-rds-for-sql-server-custom-log-shipping

  View on GitHub
  ☆12May 28, 2024Updated 2 years ago
• datamindedbe / eu-data-platform

  View on GitHub
  Spin up a minimalistic Data Analytics Platform on a European cloud provider
  ☆19Apr 22, 2026Updated 2 months ago
• itversity / mastering-postgresql

  View on GitHub
  Content related to Mastering Postgresql along with videos.
  ☆19Aug 18, 2021Updated 4 years ago
• kgn / BBBouncePass

  View on GitHub
  An Objective-C library for uploading shots to Dribbble.
  ☆13Mar 27, 2012Updated 14 years ago
• kgn / Coaches-Loupe

  View on GitHub
  Upload shots to dribbble.com
  ☆14Mar 27, 2012Updated 14 years ago
• cdpeters / dash-test-app

  View on GitHub
  A testing ground for Plotly Dash app development including app features and experimenting with dashboard visualizations.
  ☆10Oct 15, 2023Updated 2 years ago
• Managed hosting for WordPress and PHP on Cloudways • Ad
  Managed hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
• klaas / CollectionViewElCapitan

  View on GitHub
  NSCollectionView sample for OS X 10.11 ElCapitan
  ☆12Nov 24, 2017Updated 8 years ago
• Hycky / oltp-simulator

  View on GitHub
  Realistic OLTP data simulator for CDC testing with Debezium
  ☆17Nov 5, 2025Updated 8 months ago
• petereysermans / sleeptracker

  View on GitHub
  ☆10Jul 19, 2020Updated 5 years ago
• ahershy / Virtual_assistant

  View on GitHub
  Transcribe speech to text, then receive a virtual assistant response to what you say from openai
  ☆16Sep 23, 2022Updated 3 years ago
• jeremySrgt / mini-modern-data-stack

  View on GitHub
  Deploy a complete data stack in just a couple of minutes.
  ☆15Mar 6, 2024Updated 2 years ago
• jerrinss5 / Multi-threaded-Proxy-Server

  View on GitHub
  Multi-threaded simple proxy server in Python with file caching
  ☆11Oct 4, 2020Updated 5 years ago
• datarootsio / tutorial-mlops

  View on GitHub
  MLOps exercise material.
  ☆33Nov 16, 2022Updated 3 years ago
• CodeSeoul / intro-to-prefect

  View on GitHub
  ☆13Sep 23, 2023Updated 2 years ago
• osgeosuomi / qgis_plugin_tools

  View on GitHub
  ☆13Jun 30, 2026Updated last week
• Managed Database hosting by DigitalOcean • Ad
  PostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
• luchonaveiro / open-source-data-stack

  View on GitHub
  Integrating Apache Airflow, dbt, Great Expectations and Apache Superset to develop a modern open source data stack.
  ☆18Jun 19, 2022Updated 4 years ago
• DominikGolebiewski / dbt-macro-polo

  View on GitHub
  A sophisticated exploration of dbt macro capabilities, pushing the boundaries of what's possible with dbt's macro system.
  ☆18Jun 22, 2026Updated 2 weeks ago
• thanhENC / e2e-data-platform

  View on GitHub
  End-to-end data platform: A PoC Data Platform project utilizing modern data stack (Spark, Airflow, DBT, Trino, Lightdash, Hive metastore,…
  ☆49Oct 14, 2024Updated last year
• cnstlungu / postcard-company-datamart

  View on GitHub
  learning-by-doing data model built with dbt-core
  ☆17Apr 10, 2026Updated 2 months ago
• kiddojazz / CDC_Stream_Kafka_Fabric

  View on GitHub
  ☆16Apr 18, 2025Updated last year
• pixipanda / FraudDetection

  View on GitHub
  Real-time Credit card Fraud detection using Spark Streaming, Spark ML, Spark SQL, Kafka, Cassandra and Airflow
  ☆11Jul 1, 2022Updated 4 years ago
• dashbook / dashtool

  View on GitHub
  ☆17Nov 27, 2025Updated 7 months ago
• bzssm / 15-712-Advanced-and-Distributed-Operating-Systems

  View on GitHub
  CMU 15-712 lecture slides
  ☆11Jan 6, 2020Updated 6 years ago
• jess197 / football_statistics_etl_project

  View on GitHub
  ☆13Dec 28, 2023Updated 2 years ago
• Managed Kubernetes at scale on DigitalOcean • Ad
  DigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
• caio-moliveira / mcp-agents

  View on GitHub
  ☆18May 27, 2025Updated last year
• Mr-Black-Dahlia / strapi-nuxt-boilerplate

  View on GitHub
  ☆11Jan 31, 2019Updated 7 years ago
• Tejas-Haritsa-vk / Creating_Anime_Database_With_Web_Scraping

  View on GitHub
  Create an Anime database containing all the Anime currently available on the website, which includes: 'Anime Title', 'Description', 'C…
  ☆11Jun 10, 2020Updated 6 years ago
• aakashg / pm-claude-skills

  View on GitHub
  5 Claude Code skills for product managers. Drop them in your .claude/skills/ folder and go.
  ☆87Mar 4, 2026Updated 4 months ago
• siddd88 / gcp-data-engineering

  View on GitHub
  Batch Processing , orchestration using Apache Airflow and Google Workflows, spark structured Streaming and a lot more
  ☆18Jun 21, 2022Updated 4 years ago
• xianjinseow92 / Data-Science-Projects

  View on GitHub
  ☆16Apr 26, 2020Updated 6 years ago
• dpanagop / ML_and_AI_examples

  View on GitHub
  ☆10Jul 20, 2020Updated 5 years ago