juliopez / Hadoop
Infraestructura para Big Data : Hadoop + NiFi +Spark + Hive usando Docker
☆19Updated last year
Related projects ⓘ
Alternatives and complementary repositories for Hadoop
- Zeppelin docker☆15Updated 4 years ago
- Dockerizing an Apache Spark Standalone Cluster☆43Updated 2 years ago
- Creation of a data lakehouse and an ELT pipeline to enable the efficient analysis and use of data☆40Updated 11 months ago
- The demo of using Kafka, Spark, Hive, Cassandra, etc by using Docker. It produces the production ready environment for any kinds of big d…☆31Updated 5 years ago
- A docker using the airflow with Hadoop ecosystem (hive, spark, and sqoop)☆11Updated 3 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- Hadoop, Hive, Parquet and Hue in docker-compose v3☆40Updated 4 years ago
- Hadoop-Hive-Spark cluster + Jupyter on Docker☆61Updated 5 months ago
- Lecture: Big Data☆15Updated 2 weeks ago
- Big Data Docker Data Science Spark Spark3 Hadoop HDFS Scala Python Artificial Intelligence Machine Learning Jupyter Lab Notebook☆16Updated this week
- End-to-end Kafka Streaming Examples on Databricks with Evolving Avro Schemas.☆9Updated 8 months ago
- Spark and Hive docker containers sharing a common MySQL metastore☆26Updated 4 years ago
- Apche Spark Structured Streaming with Kafka using Python(PySpark)☆41Updated 5 years ago
- Design/Implement stream/batch architecture on NYC taxi data | #DE☆26Updated 3 years ago
- Developed an ETL pipeline for a Data Lake that extracts data from S3, processes the data using Spark, and loads the data back into S3 as …☆16Updated 5 years ago
- This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and …☆27Updated 9 months ago
- Slowly Changing Dimension type 2 using Hive query language using exclusive join technique with ORC Hive tables, partitioned and clustered…☆16Updated 5 years ago
- Infrastructure automation to deploy Hadoop,Hive,Spark,airflow nodes on a docker host☆20Updated 5 years ago
- ☆43Updated last year
- Delta-Lake, ETL, Spark, Airflow☆44Updated 2 years ago
- Quickly set up a POC environment for Kafka+Spark☆16Updated 7 years ago
- Spark cluster in docker containers with sample training Jupyter notebooks☆25Updated last year
- Real Time Streaming using Apache Spark Streaming [Video], published by Packt☆10Updated 2 years ago
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆14Updated 3 years ago
- Docker Big Data Tools: This docker-compose file is configured to run multiple nodes. This is a Hadoop Cluster that contains the necessary…☆28Updated 3 years ago
- Set up a 3 node spark cluster using docker containers☆33Updated 6 years ago
- A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in …☆21Updated 2 years ago
- Built a stream processing data pipeline to get data from disparate systems into a dashboard using Kafka as an intermediary.☆28Updated last year
- Workshop Big Data en Español☆22Updated last year