DirectKafka examples for Spark Streaming : 1. with checkpointing 2. Custom offset management
☆60Sep 9, 2016Updated 9 years ago
Alternatives and similar repositories for Spark-Streaming-DirectKafka-Examples
Users that are interested in Spark-Streaming-DirectKafka-Examples are comparing it to the libraries listed below
Sorting:
- ☆14Nov 3, 2016Updated 9 years ago
- SparkStreaming中利用MySQL保存Kafka偏移量保证0数据丢失☆44Aug 2, 2017Updated 8 years ago
- ☆243Jun 14, 2018Updated 7 years ago
- A light Kafka to HDFS/S3 ETL library based on Apache Spark☆40Jun 29, 2017Updated 8 years ago
- Spark structured streaming with Kafka data source and writing to Cassandra☆63Dec 5, 2019Updated 6 years ago
- 使用spring-boot-spark的一个样例☆11Aug 3, 2018Updated 7 years ago
- Java Client of the Spark Job Server implementing the arranged Rest APIs☆51Jun 4, 2021Updated 4 years ago
- 一个手动管理spark streaming集成kafka时的偏移量到zookeeper中的小项目☆133Dec 17, 2025Updated 2 months ago
- Low level integration of Spark and Kafka☆130Mar 15, 2018Updated 7 years ago
- spark流数据处理，可以从flume-ng，kafka接收数据☆11Sep 16, 2015Updated 10 years ago
- spark ml 算法原理剖析以及具体的源码实现分析☆10Jan 25, 2017Updated 9 years ago
- flink sql redis 连接器☆12Nov 20, 2023Updated 2 years ago
- springcloud-demo, 注册中心、负载均衡、熔断器、调用监控、API网关示例☆15Feb 9, 2017Updated 9 years ago
- Elasticsearch Service for Ambari☆14Feb 16, 2018Updated 8 years ago
- Ansible playbooks to help to deploy Apache Hadoop,Spark,Storm,Zookeeper,Elasticsearch,Azkaban,Flume,Hbase,Kafka,Kibana,Logstash☆10Mar 21, 2017Updated 8 years ago
- Few scripts to automate daily data loads from RDBMS to Partitioned Avro Hive table☆30Sep 25, 2014Updated 11 years ago
- Hive-JDBC-Proxy是一个高性能的HiveServer2和Spark ThriftServer的代理服务，具备负载均衡、基于规则转发Hive JDBC Client的请求给到HiveServer2和Spark ThriftServer的能力。☆33Apr 12, 2022Updated 3 years ago
- Set of ETL utils for Spark☆15May 4, 2020Updated 5 years ago
- Kafka stream for Spark with storage of the offsets in ZooKeeper☆60Apr 18, 2017Updated 8 years ago
- Clickhouse sink for akka-streams☆16Jan 3, 2022Updated 4 years ago
- 基于jsqlparser 进行sql语句解析，得到字段间的血缘关系（即所查询字段来源于哪张表中的哪个字段）☆19Oct 6, 2023Updated 2 years ago
- Kafka delivery semantics in the case of failure depend on how and when offsets are stored. Spark output operations are at-least-once. So …☆37Apr 19, 2017Updated 8 years ago
- Aloha: a distributed task scheduling and management framework☆64Dec 6, 2022Updated 3 years ago
- An example project that combines Spark Streaming, Kafka, and Parquet to transform JSON objects streamed over Kafka into Parquet files in …☆19Jun 22, 2021Updated 4 years ago
- Open-source distribute workflow schedule tools, also support streaming task.☆39Nov 11, 2017Updated 8 years ago
- Spark Structured Streaming JDBC Sink☆16Apr 26, 2021Updated 4 years ago
- A Maven-based example of using Cloudera Impala's JDBC driver☆118May 10, 2016Updated 9 years ago
- A library for querying Binlog with Apache Spark structure streaming, for Spark SQL , DataFrames and [MLSQL](https://www.mlsql.tech).☆152Apr 21, 2023Updated 2 years ago
- ☆21Jan 9, 2018Updated 8 years ago
- ☆77Oct 22, 2018Updated 7 years ago
- Flume Sink for writing events directly to parquet files in HDFS☆18May 19, 2017Updated 8 years ago
- ☆235Sep 15, 2022Updated 3 years ago
- spark实例代码☆78Nov 11, 2017Updated 8 years ago
- Write your Spark data to Kafka seamlessly☆174Jul 10, 2024Updated last year
- conbine flume,spark-streaming and redis for real-time computing☆22Oct 20, 2014Updated 11 years ago
- Spark summit 2017 europe ppt下载☆43Nov 2, 2017Updated 8 years ago
- presto 源码分析☆51Feb 2, 2018Updated 8 years ago
- Ambari service for Azkaban☆25Aug 29, 2021Updated 4 years ago
- 客户端日志埋点校验系统☆21Oct 5, 2025Updated 4 months ago