Stefen-Taime / stream-ingestion-redpanda-minioLinks
In this article, you will learn how to set up a real-time data processing and analytics environment using Docker, MySQL, Redpanda, MinIO, and Apache Spark.
☆11Updated 2 years ago
Alternatives and similar repositories for stream-ingestion-redpanda-minio
Users that are interested in stream-ingestion-redpanda-minio are comparing it to the libraries listed below
Sorting:
- Examples surrounding Databricks.☆60Updated last year
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.☆79Updated 2 years ago
- This project is for demonstrating knowledge of Data Engineering tools and concepts and also learning in the process☆47Updated 3 years ago
- Sample Data Lakehouse deployed in Docker containers using Apache Iceberg, Minio, Trino and a Hive Metastore. Can be used for local testin…☆75Updated 2 years ago
- Template for Data Engineering and Data Pipeline projects☆115Updated 2 years ago
- build dw with dbt☆49Updated last year
- Unit testing using databricks connect☆32Updated 4 years ago
- Demo of using the Nutter for testing of Databricks notebooks in the CI/CD pipeline☆152Updated last year
- Collection of Sample Databricks Spark Notebooks ( mostly for Azure Databricks )☆93Updated 7 years ago
- ☆42Updated 4 years ago
- This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which…☆104Updated 2 months ago
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆15Updated 2 years ago
- Delta-Lake, ETL, Spark, Airflow☆48Updated 3 years ago
- Sample project to demonstrate data engineering best practices☆204Updated last year
- Code for dbt tutorial☆165Updated 3 months ago
- Code samples, etc. for Databricks☆73Updated 6 months ago
- Data pipeline that scrapes Rust cheater Steam profiles☆54Updated 3 years ago
- ☆141Updated 10 months ago
- A template repository to create a data project with IAC, CI/CD, Data migrations, & testing☆280Updated last year
- Delta Lake examples☆235Updated last year
- ☆88Updated 3 years ago
- End-to-end data platform leveraging the Modern data stack☆52Updated last year
- The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for sever…☆277Updated 2 months ago
- Delta Lake helper methods in PySpark☆325Updated last year
- The resources of the preparation course for Databricks Data Engineer Professional certification exam☆160Updated last week
- Execution of DBT models using Apache Airflow through Docker Compose☆126Updated 2 years ago
- DBSQL SME Repo contains demos, tutorials, blog code, advanced production helper functions and more!☆78Updated 2 weeks ago
- Road to Azure Data Engineer Part-I: DP-200 - Implementing an Azure Data Solution☆67Updated 5 years ago
- ☆15Updated 3 years ago
- A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for …☆140Updated 5 years ago