juliopez / Hadoop
Infraestructura para Big Data : Hadoop + NiFi +Spark + Hive usando Docker
☆19Updated 2 years ago
Alternatives and similar repositories for Hadoop:
Users that are interested in Hadoop are comparing it to the libraries listed below
- Dockerizing an Apache Spark Standalone Cluster☆43Updated 2 years ago
- Creation of a data lakehouse and an ELT pipeline to enable the efficient analysis and use of data☆45Updated last year
- Zeppelin docker☆15Updated 4 years ago
- The demo of using Kafka, Spark, Hive, Cassandra, etc by using Docker. It produces the production ready environment for any kinds of big d…☆32Updated 5 years ago
- End-to-end Kafka Streaming Examples on Databricks with Evolving Avro Schemas.☆9Updated last year
- Developed an ETL pipeline for a Data Lake that extracts data from S3, processes the data using Spark, and loads the data back into S3 as …☆16Updated 5 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆54Updated last year
- Slowly Changing Dimension type 2 using Hive query language using exclusive join technique with ORC Hive tables, partitioned and clustered…☆16Updated 5 years ago
- A Flink applcation that demonstrates reading and writing to/from Apache Kafka with Apache Flink☆20Updated last year
- CICD pipeline that deploys a dbt image on a GKE cluster☆11Updated 3 years ago
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆15Updated 3 years ago
- Apache Spark using SQL☆14Updated 3 years ago
- Hadoop, Hive, Parquet and Hue in docker-compose v3☆42Updated 5 years ago
- Docker Big Data Tools: This docker-compose file is configured to run multiple nodes. This is a Hadoop Cluster that contains the necessary…☆29Updated 3 years ago
- ☆25Updated 4 years ago
- Data Engineering on GCP☆35Updated 2 years ago
- Big Data Docker Data Science Spark Spark3 Hadoop HDFS Scala Python Artificial Intelligence Machine Learning Jupyter Lab Notebook☆16Updated 3 weeks ago
- Design/Implement stream/batch architecture on NYC taxi data | #DE☆25Updated 3 years ago
- data engineering 100 days 🤖 🧲 🦾 | #DE☆40Updated last year
- Example Python and R code for Cloudera Machine Learning (CML) training☆14Updated 4 years ago
- Demonstration of using Apache Spark to build robust ETL pipelines while taking advantage of open source, general purpose cluster computin…☆24Updated last year
- A docker using the airflow with Hadoop ecosystem (hive, spark, and sqoop)☆11Updated 3 years ago
- Building Big Data Pipelines with Apache Beam, published by Packt☆86Updated 2 years ago
- Data Engineering with Scala, published by Packt☆23Updated last year
- Apche Spark Structured Streaming with Kafka using Python(PySpark)☆40Updated 5 years ago
- ☆18Updated last year
- Spark data pipeline that processes movie ratings data.☆28Updated 3 weeks ago
- Python and AirFlow - Data Pipeline Orchestration☆17Updated last year
- Spark on Kubernetes using Helm☆34Updated 4 years ago
- ☆10Updated 2 years ago