spark将hdfs数据高性能灌入kafka,然后spark streaming/structured streaming高速消费,关注性能,欢迎提供性能/代码优化建议
☆32Mar 24, 2019Updated 7 years ago
Alternatives and similar repositories for sparkStreamingKafkaPerformance
Users that are interested in sparkStreamingKafkaPerformance are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Encapsulated spark 与其他组件的结合api,方便使用,例如 es,hbase,kudu,kafka,mq等☆36Dec 18, 2019Updated 6 years ago
- Spark Structured Streaming JDBC Sink☆16Apr 26, 2021Updated 5 years ago
- Spark structured-streaming 消费kafka数据写入hbase☆33Jan 22, 2019Updated 7 years ago
- spark流数据处理,可以从flume-ng,kafka接收数据☆11Sep 16, 2015Updated 11 years ago
- 简单易用的ETL工具☆17Mar 28, 2019Updated 7 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- 封装sparkstreaming动态调节batch time(有数据就执行计算); 支持运行过程中增删topic; 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。☆181Apr 15, 2021Updated 5 years ago
- Spark Streaming example project which pulls messages from Kafka and write to HBase Table.☆11Jul 5, 2015Updated 11 years ago
- ☆14Nov 3, 2016Updated 10 years ago
- SparkStreaming项目,显示flume->Kafka->Spark->hbase(实时数据处理方案),Scala实现☆37Feb 19, 2018Updated 8 years ago
- conbine flume,spark-streaming and redis for real-time computing☆22Oct 20, 2014Updated 12 years ago
- 实时数仓的一些数据处理(mysql、canal、kafka、flink、hbase、kudu等等),以及一堆Flink的练习☆11Jul 1, 2022Updated 4 years ago
- datax-kuduwriter☆11Aug 16, 2023Updated 3 years ago
- SparkStreaming中利用MySQL保存Kafka偏移量保证0数据丢失☆43Aug 2, 2017Updated 9 years ago
- A sink to save Spark Structured Streaming DataFrame into Hive table☆29Apr 16, 2018Updated 8 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- 【易车】- Spark、flink、HBase、Hive、flume集成了一些Hadoop的原生api的一些demo(如HDFS、MapReduce:目前就这两个);同时测试一些异常功能☆16Apr 4, 2019Updated 7 years ago
- 手动管理spark streaming集成kafka的数据偏移量到zookeeper中☆21Jul 6, 2018Updated 8 years ago
- Streaming 相关项目☆15Mar 27, 2017Updated 9 years ago
- 使用spark streaming 导入kafka数据到hbase☆25Apr 14, 2016Updated 10 years ago
- java爬虫,反爬虫策略、ETL清洗数据,以及spark离线和实时分析新闻并存入ES☆19Nov 26, 2018Updated 7 years ago
- NanZi·数据服务平台是企业级 Data API 与元数据治理平台。将物理表、自定义 SQL 与语义元数据封装为可治理、可审计的 RESTful API,支持 MySQL / ClickHouse / Oracle 多源接入,提供 SQL 实验室、细粒度 RBAC 与全链…☆37Aug 27, 2026Updated 2 months ago
- 日常笔记,面试题,spark源码剖析笔记☆13May 31, 2020Updated 6 years ago
- 用户画像代码,根据算法推算出用户的性别和年龄比率☆11Dec 18, 2017Updated 8 years ago
- Spark Streaming + kafka + hbase☆15Nov 19, 2018Updated 8 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Spark.2.2源代码中文注释-配套的博客☆12Apr 11, 2018Updated 8 years ago
- Flume-to-Spark-Streaming Log Parser☆23Jun 3, 2016Updated 10 years ago
- 基于hanlp工具包的es分词插件☆10Mar 20, 2018Updated 8 years ago
- spring-boot利用scala写spark程序骨架☆28Oct 22, 2019Updated 7 years ago
- 使用shell脚本部署Apache Doris (incubating) FE & BE☆11Jul 8, 2019Updated 7 years ago
- 1.Spark离线批处理,用户实时点击统计;2.SparkSQL日志内容分析;3.受众电影分析 =>(Kafka + SparkStreaming + Redis)和(Kafka + SparkStreaming + Mysql)☆29Jun 21, 2022Updated 4 years ago
- Use Scala API to read/write data from different databases,HBase,MySQL,etc.☆24Feb 28, 2018Updated 8 years ago
- DataX 是异构数据广泛使用的离线数据同步工具/平台,实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。☆23Jan 31, 2018Updated 8 years ago
- 基于PowerCenter的数据质量监控系统☆13Dec 27, 2017Updated 8 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Apache Spark structured streaming connector for Yandex ClickHouse OLAP☆16Aug 10, 2017Updated 9 years ago
- sql实现Structured Streaming☆40Jan 4, 2019Updated 7 years ago
- 一个Spring Security添加验证码的示例☆11Sep 3, 2017Updated 9 years ago
- Code for Apache Hudi, Apache Iceberg and Delta Lake analysis☆10Feb 2, 2024Updated 2 years ago
- spark自学手册,包含了例如spark core、spark sql、spark streaming、spark-kafka、delta-lake,以及scala基础练习,还有一些例如master、shuffle源码分析,总结及翻译。☆18Jul 19, 2023Updated 3 years ago
- 基于spark streaming+flume+kafka+hbase的实时日志处理分析系统(分为控制台版本和基于springboot、Echarts等的Web UI可视化版本)☆41Jul 20, 2023Updated 3 years ago
- 蓝泰源大数据基础平台☆17Mar 7, 2018Updated 8 years ago
Are these results useful?