justinrmiller / spark-kafka-parquet-exampleLinks
An example project that combines Spark Streaming, Kafka, and Parquet to transform JSON objects streamed over Kafka into Parquet files in S3.
☆18Updated 4 years ago
Alternatives and similar repositories for spark-kafka-parquet-example
Users that are interested in spark-kafka-parquet-example are comparing it to the libraries listed below
Sorting:
- High performance HBase / Spark SQL engine☆28Updated 2 years ago
- A Spark SQL HBase connector☆29Updated 10 years ago
- A light Kafka to HDFS/S3 ETL library based on Apache Spark☆41Updated 8 years ago
- Yet Another Spark SQL JDBC/ODBC server based on the PostgreSQL V3 protocol☆34Updated 2 years ago
- ☆8Updated 7 years ago
- Spark Connector to read and write with Pulsar☆113Updated this week
- Hadoop MapReduce tool to convert Avro data files to Parquet format.☆34Updated 12 years ago
- A sink to save Spark Structured Streaming DataFrame into Hive table☆23Updated 7 years ago
- Flink Examples☆39Updated 9 years ago
- UberScriptQuery, a SQL-like DSL to make writing Spark jobs super easy☆61Updated last year
- Druid indexing plugin for using Spark in batch jobs☆101Updated 3 years ago
- Example project to show how to use Spark to read and write Avro/Parquet files☆50Updated 11 years ago
- Scripts to build a Docker image with Apache Impala with Kudu support (no HDFS needed)☆16Updated 4 years ago
- Edit code in IntelliJ, eval/run in Zeppelin notebook☆18Updated 6 years ago
- Demonstration of a Hive Input Format for Iceberg☆26Updated 4 years ago
- Spark Example using Phoenix to interact with HBase☆16Updated 8 years ago
- Demo quering counts of a event stream with Apache Flink☆23Updated 6 years ago
- ☆27Updated 4 years ago
- Source code of Blog at☆51Updated last month
- Code for processing AVRO data in Spark Streaming + Kafka (DirectKafka approach with custom offset management in zookeeper)☆29Updated 8 years ago
- Spark Streaming HBase Example☆22Updated 9 years ago
- A bridge to Apache Atlas for provenance metadata created in course of using Apache NiFi☆15Updated 2 years ago
- Java library to integrate Flink and Kudu☆54Updated 7 years ago
- Example to show how to stop the Spark Streaming Application Gracefully☆26Updated 7 years ago
- Kafka stream for Spark with storage of the offsets in ZooKeeper☆60Updated 8 years ago
- Spark 3.0.0 Structured Streaming Kafka Avro Demo☆15Updated 2 years ago
- Spark structured streaming with Kafka data source and writing to Cassandra☆62Updated 5 years ago
- Hadoop utility to compact small files☆18Updated last year
- Thoughts on things I find interesting.☆17Updated 6 months ago
- An Apache Spark app for making data movement between Apache Hive and Apache Phoenix/HBase☆14Updated 9 years ago