peopleindreamdontsleep/SparkanSpider external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

peopleindreamdontsleep/SparkanSpider

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/peopleindreamdontsleep/SparkanSpider)

Close

peopleindreamdontsleep / SparkanSpiderView on GitHubMore

• External links
• Readme badge

java爬虫，反爬虫策略、ETL清洗数据，以及spark离线和实时分析新闻并存入ES

☆19Nov 26, 2018Updated 7 years ago

Alternatives and similar repositories for SparkanSpider

Users that are interested in SparkanSpider are comparing it to the libraries listed below

• JetFeng / SohuSpider-Java

  View on GitHub
  用java写的搜狐新闻爬虫
  ☆14May 2, 2017Updated 8 years ago
• changge458 / userdraw

  View on GitHub
  用户画像代码，根据算法推算出用户的性别和年龄比率
  ☆11Dec 18, 2017Updated 8 years ago
• JasonWiki / dw_etl

  View on GitHub
  dw etl 工具 mysql 增量、全量抽取 to hive. 合并 hive 数据表, 等数据平台清洗工具
  ☆10Dec 21, 2016Updated 9 years ago
• zhoufwind / palo-deploy

  View on GitHub
  使用shell脚本部署Apache Doris (incubating) FE & BE
  ☆11Jul 8, 2019Updated 6 years ago
• crystal-tensor / spide

  View on GitHub
  网络爬虫 主要抓取的是股票数据，外汇数据，股票背景资料，股票及时新闻
  ☆12Aug 13, 2018Updated 7 years ago
• 51bigdata / StockData2Hbase

  View on GitHub
  股票交易数据处理的整个业务流程 数据源--->数据采集--->数据归类--->数据储存--->数据分析--->数据可视化
  ☆31Nov 23, 2016Updated 9 years ago
• seawaylee / spark-rec-v2

  View on GitHub
  Spark混合推荐系统大数据监控平台
  ☆11May 1, 2018Updated 7 years ago
• lxf44944 / sinaNews_crawler

  View on GitHub
  新浪新闻爬虫
  ☆15Feb 14, 2015Updated 11 years ago
• BigDataScholar / User-Portrait

  View on GitHub
  大数据【企业级360°全方位用户画像】标签开发部分源码
  ☆19Dec 18, 2020Updated 5 years ago
• a123demi / spring-sparkstreaming-kafka-10

  View on GitHub
  spring+spark streaming+kafka 10版本集成和异常问题处理
  ☆17Jul 21, 2017Updated 8 years ago
• gitzh750vp520 / OnlineEdu

  View on GitHub
  视频教育网站
  ☆17Sep 25, 2018Updated 7 years ago
• ljcan / Hive_Project

  View on GitHub
  使用Hive进行大数据分析实战
  ☆23Aug 8, 2018Updated 7 years ago
• Jax-Rene / monitor-systam-flink-quickstart

  View on GitHub
  使用flink快速构建实时监控系统报警
  ☆19Sep 7, 2019Updated 6 years ago
• DTStack / jlogstash-input-plugin

  View on GitHub
  java 版本 logstash input 插件
  ☆21Dec 20, 2018Updated 7 years ago
• cclient / kubernetes-hadoop

  View on GitHub
  k8s hadoop,在k8s上快速搭建一个hadoop/hbase/hive环境，很早的项目自已用，腾讯tbds培训，以此为基础(多了一个kafka/flink)搭一套环境练习，又捡起来了
  ☆22Mar 21, 2021Updated 4 years ago
• effiban / scala2java

  View on GitHub
  A tool for translating Scala source code into readable and maintainable Java code
  ☆13Jan 3, 2026Updated 2 months ago
• guofei1219 / SparkSQL

  View on GitHub
  SparkSQL数据分析案例
  ☆23Dec 3, 2016Updated 9 years ago
• JThink / teemo

  View on GitHub
  spring-boot利用scala写spark程序骨架
  ☆28Oct 22, 2019Updated 6 years ago
• 344399160 / sparkForDB

  View on GitHub
  使用spark对hive、hbase、ES的读写， 实现一次配置可对不同数据库进行导入导出，并对ES、hbase进行封装
  ☆32May 6, 2017Updated 8 years ago
• wushu0725 / codegen

  View on GitHub
  springboot+vue页面 REST代码生成器，生成单表的增删改查
  ☆31May 14, 2023Updated 2 years ago
• peopleindreamdontsleep / StructuredStreamingInSQL

  View on GitHub
  sql实现Structured Streaming
  ☆39Jan 4, 2019Updated 7 years ago
• JP6907 / RealTimeLogAnalyze

  View on GitHub
  一个大数据实时流处理日志分析系统 Demo
  ☆30Nov 16, 2022Updated 3 years ago
• RipleyGit / snailgirl

  View on GitHub
  使用springboot+mybatis后台框架 前端bootstrap框架 添加web socket实时提醒订单消息 使用springsecurity进行权限拦截 邮箱+短信验证 ，echart图表显示用户订单信息，poi表报打印等等。。。
  ☆12Oct 25, 2017Updated 8 years ago
• Troy-Wang / SpringBatch_Gringotts

  View on GitHub
  A batch-processing system base on Spring Boot and Spring Batch. 一个基于SpringBoot和SpringBatch的批处理系统。
  ☆10Sep 10, 2018Updated 7 years ago
• MicroCountry / ddy-stock-real

  View on GitHub
  股票行情--来源恒生，新浪，腾讯，网易
  ☆32Oct 16, 2018Updated 7 years ago
• fullstackyang / weibo-crawler

  View on GitHub
  新型的免登录微博爬虫，自动获取Cookie直接进行抓取和解析微博数据，免去了账号登录的过程，彻底摆脱账号被封的困扰
  ☆36Oct 15, 2017Updated 8 years ago
• smartloli / hive-cube

  View on GitHub
  Data self exporting and monitoring platform based on Hive data warehouse. https://hc.smartloli.org
  ☆36Jul 28, 2017Updated 8 years ago
• RickyHuo / elasticsearch-analysis-pinyin

  View on GitHub
  This Pinyin Analysis plugin is used to do conversion between Chinese characters and Pinyin.
  ☆10Mar 28, 2019Updated 6 years ago
• YeliangQiu / Huawei2019DP

  View on GitHub
  华为软件精英挑战赛2019，实时计算全图路况，每辆车在每个时刻(或隔几个时刻)根据自身信息生成自己的权重矩阵，利用SPFA算法动态规划路径
  ☆10Apr 14, 2019Updated 6 years ago
• positivepeng / NovelSpider

  View on GitHub
  晋江文学城大数据分析项目
  ☆11Jul 6, 2019Updated 6 years ago
• wjhwangjinhui / wenshu

  View on GitHub
  ☆12Jan 5, 2019Updated 7 years ago
• luseiee / machineLearningWithSpark

  View on GitHub
  Spark projects. Learning book "Machine Learning with Spark"
  ☆10Jun 3, 2017Updated 8 years ago
• ex34toway / uw-log-es

  View on GitHub
  一个基于ElasticSearch的业务日志记录工具
  ☆10Nov 5, 2018Updated 7 years ago
• madpudding / RelationshipCrawler

  View on GitHub
  知网、万方、专利局爬虫
  ☆11Mar 20, 2019Updated 6 years ago
• U3DC / Unity_UpLoader

  View on GitHub
  A solution for UpLoad data(TXT,ScreenShot) to server,Contact with PHP ( 数据上传解决方案，比如上传log信息，上传屏幕截图，PHP后端交互存储文件）
  ☆11Aug 22, 2018Updated 7 years ago
• Martin-030621 / TouTiao_Selenium

  View on GitHub
  今日头条搜索引擎以及新闻详情页爬虫（Selenium）
  ☆15Mar 13, 2025Updated 11 months ago
• HallBlazzar / VGG16ConvolutionalAutoEncoder

  View on GitHub
  ☆10Jun 26, 2018Updated 7 years ago
• herywang / jeesite4

  View on GitHub
  襄阳智慧交通大数据平台建设
  ☆15Jun 25, 2022Updated 3 years ago
• xiaozhch5 / spring-guides

  View on GitHub
  ☆11Sep 1, 2022Updated 3 years ago