A real-time streaming ETL pipeline for streaming and performing sentiment analysis on Twitter data using Apache Kafka, Apache Spark and Delta Lake.
☆29Aug 8, 2020Updated 5 years ago
Alternatives and similar repositories for spark-twitter-streaming
Users that are interested in spark-twitter-streaming are comparing it to the libraries listed below
Sorting:
- 😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS☆51Aug 23, 2019Updated 6 years ago
- This repo contains Big Data Project, its about "Real Time Twitter Sentiment Analysis via Kafka, Spark Streaming, MongoDB and Django Dashb…☆40May 20, 2024Updated last year
- Spark data pipeline that processes movie ratings data.☆31Updated this week
- Chat with PDF locally: An advanced chatbot using Ollama / Sambanova LLMs to interactively extract information from PDFs, Using Streamlit …☆17May 17, 2025Updated 9 months ago
- Docker compose and Google Colab demo to build a CDC with Delta Lake☆15Sep 7, 2022Updated 3 years ago
- A bunch of crawlers for extracting data from various sites (site name is mentioned for each one)☆11May 2, 2024Updated last year
- Modeling and Simulation in Python and MATLAB/Octave☆12Jun 25, 2021Updated 4 years ago
- ☆10Feb 5, 2026Updated last month
- ☆11Mar 28, 2022Updated 3 years ago
- ☆10Nov 28, 2020Updated 5 years ago
- Dockerfile and artifacts for running a self-contained HDP 2.3 "cluster" in a docker container☆10Aug 30, 2016Updated 9 years ago
- Delta-Lake, ETL, Spark, Airflow☆48Oct 9, 2022Updated 3 years ago
- Video streaming with kafka☆10Sep 23, 2023Updated 2 years ago
- A bunch of low-level basic methods for data processing and monitoring with Scala Spark☆10Jun 29, 2018Updated 7 years ago
- ☆11Aug 3, 2019Updated 6 years ago
- Processing TfL data for bike usage with Google Cloud Platform.☆46Jul 15, 2022Updated 3 years ago
- A consumer of a Kafka topic based on Flink☆12Oct 5, 2022Updated 3 years ago
- Simple log parsing example in Python☆14Oct 7, 2015Updated 10 years ago
- Test consul cluster on docker swarm cluster (by devteds.com)☆14Mar 11, 2018Updated 7 years ago
- InfluxDB 2 Connector for Kafka☆13Mar 6, 2020Updated 6 years ago
- Explore building an advanced infrastructure for enhancing QuantConnect with Snowflake, Databricks, Airflow & AWS. Learn the basics of qua…☆15Jan 27, 2024Updated 2 years ago
- A Docker Compose Consul network definition☆11Mar 2, 2018Updated 8 years ago
- StackHPC Kayobe configuration☆20Updated this week
- This repo demonstrates how to use AWS application auto-scaling to implement custom-scaling in your Kinesis Data Analytics for Apache Flin…☆19Feb 21, 2025Updated last year
- ☆15Jul 31, 2022Updated 3 years ago
- Implementation of Neural Networks with Python☆12Jul 5, 2020Updated 5 years ago
- This repository contains all tutorials for Apache Spark, Delta Lake, Koalas, MLflow, and other.☆15May 29, 2020Updated 5 years ago
- Social Media Analysis, scalable solution, flexible deployment that analyses social media contents☆10Jul 20, 2023Updated 2 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆56May 6, 2023Updated 2 years ago
- This provider contains operators, decorators and triggers to send a ray job from an airflow task☆24Oct 27, 2025Updated 4 months ago
- A repo to track data engineering projects☆13Nov 11, 2022Updated 3 years ago
- ☆16May 1, 2023Updated 2 years ago
- Empirical Models for the Realistic Generation of Cooperative Awareness Messages in Vehicular Networks☆14Apr 23, 2020Updated 5 years ago
- Spark on Kubernetes samples☆20Jun 8, 2021Updated 4 years ago
- Correlation matrix with scatter plot using d3.js☆19Nov 5, 2014Updated 11 years ago
- Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark☆17Mar 2, 2023Updated 3 years ago
- Insight Data Engineering project: A platform built in HDFS, Spark and Airflow to help you to find social influencers from GitHub Net…☆16May 21, 2024Updated last year
- 🏠 A Data Science Project done to find the factors that most affect the price of an Airbnb listing.☆23Aug 7, 2020Updated 5 years ago
- Decrypts and displays the seed from an Electrum (1.x, 2.x, or an Electrum-LTC) wallet file, providing detailed error messages if required…☆17Dec 30, 2021Updated 4 years ago