used Airflow, Postgres, Kafka, Spark, and Cassandra, and GitHub Actions to establish an end-to-end data pipeline
☆33Oct 25, 2023Updated 3 years ago
Alternatives and similar repositories for Data-Streaming-Project
Users that are interested in Data-Streaming-Project are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- End-to-end data pipeline that ingests, processes, and stores data. It uses Apache Airflow to schedule scripts that fetch data from an API…☆23Sep 23, 2026Updated last month
- ☆10Mar 11, 2023Updated 3 years ago
- End-to-end ELT data engineering project☆23Dec 24, 2022Updated 3 years ago
- Code for the Data Engineering Zoomcamp☆20Dec 12, 2022Updated 3 years ago
- ☆23May 13, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Ecommerce Realtime Data Pipeline (Data Modeling, Workflow Orchestration, Change Data Capture, Analytical Database and Dashboarding)☆72Mar 9, 2024Updated 2 years ago
- TFT (Training from Traces) benchmark: 5 scenarios comparing TFT pipeline vs Direct Training on SGD restaurant traces☆13Sep 7, 2026Updated 2 months ago
- ☆12Mar 6, 2021Updated 5 years ago
- A dockerized 5 min. mini-batch data pipeline. Spark Structured Streaming: reading from Kafka to PostgreSQL as a sink DB. Airflow, Kafka, …☆10Jul 17, 2024Updated 2 years ago
- ☆15May 13, 2024Updated 2 years ago
- Creation of a data lakehouse and an ELT pipeline to enable the efficient analysis and use of data☆50Dec 2, 2023Updated 2 years ago
- This project aims to move the data from a Relational database system (RDBMS) to a Hadoop file system (HDFS)☆11Apr 29, 2022Updated 4 years ago
- A data pipeline moving data from a Relational database system (RDBMS) to a Hadoop file system (HDFS).☆15Jun 3, 2021Updated 5 years ago
- A well-documented explanation of data structure types including Linked List, Hash table, Binary Tree, Queues, Stack☆13Jul 30, 2022Updated 4 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- This is a simple object detection application, which can be used to detect military objects☆15Nov 16, 2020Updated 6 years ago
- Fully dockerized Data Warehouse (DWH) using Airflow, dbt, PostgreSQL and dashboard using redash☆27Nov 12, 2022Updated 4 years ago
- A sophisticated trading system leveraging local LLM deployment through Ollama, distributed computing with Apache Spark, and vector-based …☆15Feb 3, 2025Updated last year
- ☆12Apr 26, 2021Updated 5 years ago
- ☆24Dec 4, 2023Updated 2 years ago
- ☆15Mar 14, 2024Updated 2 years ago
- Computer vision based technologies implemented using tensorflow from ground up☆16Nov 12, 2026Updated last week
- Big data cluster for training/teaching purposes☆15May 5, 2026Updated 6 months ago
- ☆17Jul 18, 2022Updated 4 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Glue ETL job or EMR Spark that gets from data catalog, modifies and uploads to S3 and Data Catalog☆13Aug 26, 2023Updated 3 years ago
- A system for reading scanned documents and grouping them into high level topics☆14Aug 4, 2020Updated 6 years ago
- An open library of reusable AI skills for procurement, sourcing, contracting and supplier management.☆21Jul 27, 2026Updated 3 months ago
- ☆21Jun 10, 2023Updated 3 years ago
- City-GAN is a Generative Adversarial Neural Network that generates typical facades of various cities around the world.☆20Mar 22, 2022Updated 4 years ago
- A platform that helps developers to better understand CSS through declaration interpretation and may even improve them through suggestion…☆14Jul 3, 2021Updated 5 years ago
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆20Aug 12, 2025Updated last year
- Use YOLOv8 with a DJI Tello Drone! Controlled with the Tello app and provides video results post-flight.☆20Dec 8, 2023Updated 2 years ago
- It is a assemble to include all Practice Projects about Big Data Topic, includes Hadoop, Spark, Spark Streaming and Kafka☆11Mar 7, 2019Updated 7 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Covid Doctor chatbot using DialoGPT☆13May 25, 2022Updated 4 years ago
- ☆40Mar 13, 2026Updated 8 months ago
- This is end-to-end-recommender-system repo has full recommender system implementation from collecting data, modeling and deploying machin…☆14May 19, 2021Updated 5 years ago
- ☆20Jan 23, 2023Updated 3 years ago
- Data-Science-Projects-in-Python☆11Jul 25, 2018Updated 8 years ago
- ☆19Jun 22, 2022Updated 4 years ago
- An End-to-End ETL data pipeline that leverages pyspark parallel processing to process about 25 million rows of data coming from a SaaS ap…☆27Dec 7, 2022Updated 3 years ago
Are these results useful?