spark将hdfs数据高性能灌入kafka，然后spark streaming/structured streaming高速消费，关注性能，欢迎提供性能/代码优化建议
☆32Mar 24, 2019Updated 7 years ago
Alternatives and similar repositories for sparkStreamingKafkaPerformance
Users that are interested in sparkStreamingKafkaPerformance are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 简单易用的ETL工具☆17Mar 28, 2019Updated 7 years ago
- Encapsulated spark 与其他组件的结合api，方便使用，例如 es，hbase，kudu，kafka，mq等☆36Dec 18, 2019Updated 6 years ago
- Spark Streaming example project which pulls messages from Kafka and write to HBase Table.☆11Jul 5, 2015Updated 10 years ago
- ☆14Nov 3, 2016Updated 9 years ago
- SparkStreaming项目，显示flume->Kafka->Spark->hbase(实时数据处理方案)，Scala实现☆36Feb 19, 2018Updated 8 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- spark流数据处理，可以从flume-ng，kafka接收数据☆11Sep 16, 2015Updated 10 years ago
- 封装sparkstreaming动态调节batch time(有数据就执行计算)； 支持运行过程中增删topic； 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。☆181Apr 15, 2021Updated 5 years ago
- Spark structured streaming examples with using of version 3.5.1☆27Apr 27, 2024Updated 2 years ago
- datax-kuduwriter☆11Aug 16, 2023Updated 2 years ago
- SparkStreaming中利用MySQL保存Kafka偏移量保证0数据丢失☆43Aug 2, 2017Updated 8 years ago
- 实时数仓的一些数据处理（mysql、canal、kafka、flink、hbase、kudu等等），以及一堆Flink的练习☆11Jul 1, 2022Updated 3 years ago
- 【易车】- Spark、flink、HBase、Hive、flume集成了一些Hadoop的原生api的一些demo（如HDFS、MapReduce：目前就这两个）；同时测试一些异常功能☆16Apr 4, 2019Updated 7 years ago
- Spark structured-streaming 消费kafka数据写入hbase☆33Jan 22, 2019Updated 7 years ago
- 新零售大数据平台-运维监控平台的开发☆14Jan 14, 2019Updated 7 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- 手动管理spark streaming集成kafka的数据偏移量到zookeeper中☆21Jul 6, 2018Updated 7 years ago
- Streaming 相关项目☆15Mar 27, 2017Updated 9 years ago
- conbine flume,spark-streaming and redis for real-time computing☆22Oct 20, 2014Updated 11 years ago
- Jedis-x is a Redis Java client base on Jedis. It was primarily built to detect and remove the failed node automatically, also add the fai…☆10Jan 31, 2015Updated 11 years ago
- 1.Spark离线批处理，用户实时点击统计；2.SparkSQL日志内容分析；3.受众电影分析 =>（Kafka + SparkStreaming + Redis）和（Kafka + SparkStreaming + Mysql）☆29Jun 21, 2022Updated 3 years ago
- 基于PowerCenter的数据质量监控系统☆13Dec 27, 2017Updated 8 years ago
- DataX 是异构数据广泛使用的离线数据同步工具/平台，实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。☆23Jan 31, 2018Updated 8 years ago
- Use Scala API to read/write data from different databases,HBase,MySQL,etc.☆24Feb 28, 2018Updated 8 years ago
- Flume-to-Spark-Streaming Log Parser☆23Jun 3, 2016Updated 10 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Apache Spark structured streaming connector for Yandex ClickHouse OLAP☆16Aug 10, 2017Updated 8 years ago
- 使用spark streaming 导入kafka数据到hbase☆25Apr 14, 2016Updated 10 years ago
- spark自学手册，包含了例如spark core、spark sql、spark streaming、spark-kafka、delta-lake，以及scala基础练习，还有一些例如master、shuffle源码分析，总结及翻译。☆18Jul 19, 2023Updated 2 years ago
- ☆12May 11, 2016Updated 10 years ago
- Examples of Integrating Spark Streaming, Flume, and HBase to solve Streaming problems☆18Feb 27, 2014Updated 12 years ago
- Code for Apache Hudi, Apache Iceberg and Delta Lake analysis☆10Feb 2, 2024Updated 2 years ago
- A maven project with codes of the book "Hadoop: The Definitive Guide(Fourth Edition)"☆10Jul 17, 2023Updated 2 years ago
- TPC-DS benchmark kit with some modifications/additions☆10Nov 12, 2015Updated 10 years ago
- The open source implement of AceRanking.☆15Apr 10, 2020Updated 6 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- https://docs.byzer.org☆11Aug 8, 2024Updated last year
- Gateway platform based on spring-cloud-gateway enhanced version.☆13Jun 4, 2021Updated 5 years ago
- Demonstrate the some of features of gRPC☆14Dec 15, 2019Updated 6 years ago
- flink实时处理kafka传来的数据通过连接池技术写入hbase☆95Jul 18, 2022Updated 3 years ago
- simple inverted index full text search engine written in python☆13Oct 3, 2013Updated 12 years ago
- sql实现Structured Streaming☆39Jan 4, 2019Updated 7 years ago
- elasticsearch7.9 cdh-ext-parcels and single machine multi instance☆10Jul 12, 2021Updated 4 years ago