Get Crypto data from API, stream it to Kafka with Airflow. Write data to MySQL and visualize with Metabase
☆17Oct 2, 2023Updated 3 years ago
Alternatives and similar repositories for crypto_api_kafka_airflow_streaming
Users that are interested in crypto_api_kafka_airflow_streaming are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Glue ETL job or EMR Spark that gets from data catalog, modifies and uploads to S3 and Data Catalog☆13Aug 26, 2023Updated 3 years ago
- Writes the CSV file to Postgres, read table and modify it. Write more tables to Postgres with Airflow.☆38Sep 1, 2023Updated 3 years ago
- Docker Apache Airflow☆13Mar 1, 2023Updated 3 years ago
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆66Jul 21, 2023Updated 3 years ago
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆145Jul 27, 2023Updated 3 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆43Sep 26, 2023Updated 3 years ago
- Data pipeline to build a data warehouse on Postgres☆16Aug 11, 2024Updated 2 years ago
- ☆13Mar 5, 2023Updated 3 years ago
- A project portfolio to accompany my resume☆32Sep 5, 2023Updated 3 years ago
- RedditR for Content Engagement and Recommendation☆18Dec 21, 2017Updated 8 years ago
- This is a collecton of Amazon CDK projects to show how to directly ingest streaming data from Amazon Mananged Service for Apache Kafka (M…☆16Jul 28, 2026Updated 3 months ago
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 3 years ago
- ☆16Oct 8, 2025Updated last year
- ☆20Mar 11, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- how to unit test your PySpark code☆29Mar 26, 2021Updated 5 years ago
- ☆28Oct 14, 2024Updated 2 years ago
- This solution helps you deploy ETL processes and data storage resources to create an Insurance Lake using Amazon S3 buckets for storage, …☆21Feb 5, 2026Updated 9 months ago
- Databricks ML in Action, Published by Packt☆35Mar 2, 2026Updated 8 months ago
- Use a AWS Glue Python Shell Job to connect to your Amazon Redshift cluster and execute a SQL script stored in Amazon S3.☆21Aug 8, 2022Updated 4 years ago
- Welcome to my data engineering projects repository! Here you will find a collection of data engineering projects that I have worked on.☆27Apr 27, 2023Updated 3 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆50Dec 11, 2023Updated 2 years ago
- A fully serverless, event-driven data pipeline that ingests, enriches, validates, and visualizes real-time news data using AWS services. …☆26Aug 10, 2025Updated last year
- Some example projects for Data Engineers to build, end-to-end.☆39Nov 8, 2023Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- [Archived 2026-04-09] Unmaintained. [not maintained as of 2025] General tooling helpers for simplifying cross repository automation using…☆13Apr 9, 2026Updated 6 months ago
- Prometheus exporter for Amazon Smart Air Quality Monitor☆12Jan 21, 2022Updated 4 years ago
- Python Pydantic model of the Prometheus Alertmanager alert payload☆10Jul 24, 2023Updated 3 years ago
- Data pipeline and meta-analysis dashboard for IT job postings from the web.☆11Jan 31, 2025Updated last year
- Inventory is an application that tracks deployed services/containers. It was built with a homelab in mind.☆16Jun 9, 2025Updated last year
- Telegram bot that manages creation of queues / attendance lists for periodic events.☆15Dec 23, 2024Updated last year
- Playable synthesizer created with Tone.js, Next.js, and React.☆10Aug 14, 2022Updated 4 years ago
- ChatOps for Tinkerbell☆15Jun 11, 2020Updated 6 years ago
- this is a simple sso for devops api☆12May 31, 2014Updated 12 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- A template for Python projects that need to use a relational database, including tooling for managing schema migrations and testing again…☆13Dec 13, 2024Updated last year
- Cli Prometheus metrics viewer.☆13May 3, 2023Updated 3 years ago
- ryantoken.com v3 - written in Svelte, Tailwind, and mdsvex. Replaced by ryantoken.com v4☆11Jul 27, 2025Updated last year
- Framework to enforce long term health of your AWS Data Lake by providing visibility into operational, data quality and business metrics.☆31Aug 19, 2021Updated 5 years ago
- End-to-end data platform leveraging the Modern data stack☆53Apr 10, 2024Updated 2 years ago
- An extremely simple LinkedIn composer for the sole purpose of crafting posts☆11Updated this week
- Hello World! from tiny app to Kubernetes with CI/CD☆10May 16, 2023Updated 3 years ago