SelfEvoAI/sparkStreamingKafkaPerformance external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

SelfEvoAI/sparkStreamingKafkaPerformance

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/SelfEvoAI/sparkStreamingKafkaPerformance)

Close

SelfEvoAI / sparkStreamingKafkaPerformanceView on GitHubMore

• External links
• Readme badge

spark将hdfs数据高性能灌入kafka，然后spark streaming/structured streaming高速消费，关注性能，欢迎提供性能/代码优化建议

☆32Mar 24, 2019Updated 7 years ago

Alternatives and similar repositories for sparkStreamingKafkaPerformance

Users that are interested in sparkStreamingKafkaPerformance are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• mshtelma / spark-structured-streaming-jdbc-sink

  View on GitHub
  Spark Structured Streaming JDBC Sink
  ☆16Apr 26, 2021Updated 5 years ago
• dxer / dataLink

  View on GitHub
  简单易用的ETL工具
  ☆17Mar 28, 2019Updated 7 years ago
• LinMingQiang / spark-utils

  View on GitHub
  Encapsulated spark 与其他组件的结合api，方便使用，例如 es，hbase，kudu，kafka，mq等
  ☆36Dec 18, 2019Updated 6 years ago
• Gschiavon / Kafka-SparkStreaming-HDFS

  View on GitHub
  ☆14Nov 3, 2016Updated 9 years ago
• MrXuSS / RealTimeDW

  View on GitHub
  实时数仓的一些数据处理（mysql、canal、kafka、flink、hbase、kudu等等），以及一堆Flink的练习
  ☆11Jul 1, 2022Updated 4 years ago
• Managed Database hosting by DigitalOcean • Ad
  PostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
• flowers2023 / spark-streaming

  View on GitHub
  spark流数据处理，可以从flume-ng，kafka接收数据
  ☆11Sep 16, 2015Updated 10 years ago
• LinMingQiang / sparkstreaming

  View on GitHub
  封装sparkstreaming动态调节batch time(有数据就执行计算)； 支持运行过程中增删topic； 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。
  ☆181Apr 15, 2021Updated 5 years ago
• AndrewKuzmin / spark-structured-streaming-examples

  View on GitHub
  Spark structured streaming examples with using of version 3.5.1
  ☆27Apr 27, 2024Updated 2 years ago
• Mr-KIDBK / datax-kudu-plugins

  View on GitHub
  datax-kuduwriter
  ☆11Aug 16, 2023Updated 2 years ago
• E-SoulDataGroup / spark_streaming_kafka_offset

  View on GitHub
  SparkStreaming中利用MySQL保存Kafka偏移量保证0数据丢失
  ☆43Aug 2, 2017Updated 9 years ago
• duhanmin / structured-streaming-Kafka2HBase

  View on GitHub
  Spark structured-streaming 消费kafka数据写入hbase
  ☆33Jan 22, 2019Updated 7 years ago
• wangfengchao / sparkStreaming-offset-to-zk

  View on GitHub
  手动管理spark streaming集成kafka的数据偏移量到zookeeper中
  ☆21Jul 6, 2018Updated 8 years ago
• gh-gan / SparkStreaming

  View on GitHub
  Streaming 相关项目
  ☆15Mar 27, 2017Updated 9 years ago
• EdwardsBean / flume-spark-streaming

  View on GitHub
  conbine flume,spark-streaming and redis for real-time computing
  ☆22Oct 20, 2014Updated 11 years ago
• Proton VPN Special Offer - Get 70% off • Ad
  Special partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
• looker / spark_log_data

  View on GitHub
  Flume-to-Spark-Streaming Log Parser
  ☆23Jun 3, 2016Updated 10 years ago
• dtinfor / dqcenter

  View on GitHub
  基于PowerCenter的数据质量监控系统
  ☆13Dec 27, 2017Updated 8 years ago
• Higmin / Spark-Learning

  View on GitHub
  1.Spark离线批处理，用户实时点击统计；2.SparkSQL日志内容分析；3.受众电影分析 =>（Kafka + SparkStreaming + Redis）和（Kafka + SparkStreaming + Mysql）
  ☆29Jun 21, 2022Updated 4 years ago
• Arvin-Mark / DataX-src

  View on GitHub
  DataX 是异构数据广泛使用的离线数据同步工具/平台，实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。
  ☆23Jan 31, 2018Updated 8 years ago
• DmitryBe / spark-streaming-clickhouse

  View on GitHub
  Apache Spark structured streaming connector for Yandex ClickHouse OLAP
  ☆16Aug 10, 2017Updated 9 years ago
• TopSpoofer / kafka-spark-streaming-to-hbase

  View on GitHub
  使用spark streaming 导入kafka数据到hbase
  ☆25Apr 14, 2016Updated 10 years ago
• sev7e0 / wow-spark

  View on GitHub
  spark自学手册，包含了例如spark core、spark sql、spark streaming、spark-kafka、delta-lake，以及scala基础练习，还有一些例如master、shuffle源码分析，总结及翻译。
  ☆18Jul 19, 2023Updated 3 years ago
• oliversavio / hudi-data-lake-example

  View on GitHub
  ☆10Apr 13, 2020Updated 6 years ago
• bartosz25 / acid-file-formats

  View on GitHub
  Code for Apache Hudi, Apache Iceberg and Delta Lake analysis
  ☆10Feb 2, 2024Updated 2 years ago
• GPUs on demand by Runpod - Special Offer Available • Ad
  Run AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
• hongs01 / ltybdservice-root

  View on GitHub
  蓝泰源大数据基础平台
  ☆17Mar 7, 2018Updated 8 years ago
• huydx / fulltext_engine

  View on GitHub
  simple inverted index full text search engine written in python
  ☆13Oct 3, 2013Updated 12 years ago
• duhanmin / kafka-flink-hbase

  View on GitHub
  flink实时处理kafka传来的数据通过连接池技术写入hbase
  ☆95Jul 18, 2022Updated 4 years ago
• tzutalin / example-grpc

  View on GitHub
  Demonstrate the some of features of gRPC
  ☆14Dec 15, 2019Updated 6 years ago
• yzxie / easy-log

  View on GitHub
  分布式日志分析与性能监控服务，集成flume，kafka，netty，websocket，dubbo，redis，hbase等
  ☆12Oct 11, 2023Updated 2 years ago
• peopleindreamdontsleep / StructuredStreamingInSQL

  View on GitHub
  sql实现Structured Streaming
  ☆40Jan 4, 2019Updated 7 years ago
• JINGXI6 / elasticsearch-cdh-parcels

  View on GitHub
  elasticsearch7.9 cdh-ext-parcels and single machine multi instance
  ☆10Jul 12, 2021Updated 5 years ago
• lj-michale / bigdata-learning

  View on GitHub
  大数据学习：各种计算框架：
  ☆11May 20, 2022Updated 4 years ago
• WeiWan5675 / Flink-DSP

  View on GitHub
  一个基于Flink的数据流业务处理平台
  ☆26Oct 9, 2022Updated 3 years ago
• Serverless GPU API endpoints on Runpod - Get Bonus Credits • Ad
  Skip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
• Jiede1 / Spark_Programming_basis

  View on GitHub
  Spark编程基础教程，参考资料，囊括整个spark生态组件的学习，基于Scala；源自厦大实验室大数据教材
  ☆15Feb 9, 2020Updated 6 years ago
• JThink / teemo

  View on GitHub
  spring-boot利用scala写spark程序骨架
  ☆28Oct 22, 2019Updated 6 years ago
• mijingling / springBoot-MQ-active

  View on GitHub
  springboot+activemq
  ☆12Jan 25, 2017Updated 9 years ago
• isaac-taylor / BloomFilter

  View on GitHub
  A simple Bloom Filter implementation in Java
  ☆16Oct 21, 2012Updated 13 years ago
• haozhang-x / spark-waimai

  View on GitHub
  基于spark的外卖大数据平台分析系统
  ☆47Dec 16, 2018Updated 7 years ago
• tygxy / StreamingDemo

  View on GitHub
  Spark Streaming + kafka + hbase
  ☆15Nov 19, 2018Updated 7 years ago
• DataSky / HData

  View on GitHub
  一个多数据源的数据传输工具
  ☆29Apr 23, 2014Updated 12 years ago