The Data Pipeline and Analytics Stack is a comprehensive solution designed for processing, storing, and visualizing data. Explore a complete data pipeline with all components seamlessly set up and ready to use
☆18Dec 26, 2023Updated 2 years ago
Alternatives and similar repositories for bigdata-ETL-pipeline
Users that are interested in bigdata-ETL-pipeline are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Building Data Lakehouse by open source technology. Support end to end data pipeline, from source data on AWS S3 to Lakehouse, visualize a…☆42Dec 15, 2025Updated 8 months ago
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆21Aug 12, 2025Updated last year
- 📚🧪 Traffic Sentinel is a learning-focused POC that explores a scalable IoT architecture using Fog nodes and Apache Flink to process 📷 …☆28Dec 29, 2025Updated 8 months ago
- Run an open-source data LakeHouse locally using Docker Compose☆12May 31, 2024Updated 2 years ago
- Files for the Docker and Kubernetes on Google Cloud Hands-On labs☆11Mar 14, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆13Dec 3, 2021Updated 4 years ago
- Data Engineering & Analysis Project- San Francisco Eviction Data ETL Pipeline An end-to-end batch data pipeline for performing ETL on San…☆10Oct 2, 2025Updated 11 months ago
- KnetBuilder data integration platform for building knowledge graphs. Previously known as ondex.☆16Apr 2, 2026Updated 5 months ago
- trino monitoring with JMX metrics through Prometheus and Grafana☆18Aug 14, 2024Updated 2 years ago
- Gradient Boosting Models on Real-Time Sensor Data for AI-Enhanced Vehicle Predictive Maintenance. By using a web-based interface to forec…☆19Nov 17, 2024Updated last year
- This project implements an ELT (Extract - Load - Transform) data pipeline with the goodreads dataset, using dagster (orchestration), spar…☆45Apr 22, 2023Updated 3 years ago
- Patterns and concepts for building resilient data pipelines in Python and Scala☆16Aug 27, 2024Updated 2 years ago
- Predictive Incident Management analyses large data sets to identify risk patterns, predict outcomes, and guide teams on effective decisio…☆16Nov 25, 2022Updated 3 years ago
- Jupyter notebooks for the teaching of mechanics☆11Oct 8, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Automate data collection from Spotify's worldwide ranking in 50+ countries☆25May 3, 2020Updated 6 years ago
- Sample Data Lakehouse deployed in Docker containers using Apache Iceberg, Minio, Trino and a Hive Metastore. Can be used for local testin…☆82Sep 2, 2023Updated 3 years ago
- Jena Plugins for Pentaho Kettle to transform data into RDF☆12May 31, 2023Updated 3 years ago
- Template for projects in PyTorch powered with PyTorch Lightning, Telegrad and MLflow. Get updates on mobile and streamline PyTorch code f…☆10May 1, 2023Updated 3 years ago
- Sample code and documentation for very basic things that I can't remember but want to aggregate in one place☆13Nov 7, 2021Updated 4 years ago
- ☆14Mar 24, 2021Updated 5 years ago
- A simple Perceptron in Python☆10Feb 11, 2022Updated 4 years ago
- Crawling the data from lazada, websosanh, compare.vn, cdiscount and cungmua with flexible configs☆30Jul 7, 2016Updated 10 years ago
- Extension giúp ta tự động điền form Quy chế + Pháp Luật - HUST☆23Aug 11, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Hands on workshop "Refactor your Jupyter notebooks into maintainable data science code with Kedro"☆18Jan 22, 2025Updated last year
- Spark and Hive docker containers sharing a common MySQL metastore☆26Apr 17, 2020Updated 6 years ago
- Completion of three Deep Q-Networks : Deep Q-Network (DQN), Double Deep Q-Network (DDQN), Double Dueling Deep Q-Network (D3QN)☆12Jun 26, 2021Updated 5 years ago
- ☆19Feb 28, 2022Updated 4 years ago
- Skribify is a powerful transcription and summarization tool that leverages the power of OpenAI's GPT-4 and WhisperAI to generate concise …☆12Aug 6, 2026Updated last month
- A Retail store management system - DBMS project (Sep 2015) written in Django (Python)☆11Aug 11, 2020Updated 6 years ago
- Python implementation of binary max-heaps.☆11Mar 22, 2020Updated 6 years ago
- Doing sql in notebooks.☆15Aug 14, 2023Updated 3 years ago
- ☆10Jan 31, 2021Updated 5 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Ansible collection to deploy the components of TDP☆23Updated this week
- ⚡ FutureGPT - Application development framework that connects GPT-4 with external data, the internet, other applications and language mod…☆13May 14, 2023Updated 3 years ago
- ☆13Jun 21, 2021Updated 5 years ago
- Source code for Deep Reinforcement Learning for Real-Time Optimization of Pumps in Water Distribution Systems paper.☆24May 25, 2023Updated 3 years ago
- ☆12Mar 17, 2022Updated 4 years ago
- Kafka and Spark Integration. Alll code in maven project.☆14Jun 30, 2026Updated 2 months ago
- Web app designed to enhance your interaction with OpenAI's language models☆12Jun 14, 2023Updated 3 years ago