peopleindreamdontsleep/SparkanSpider external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

peopleindreamdontsleep/SparkanSpider

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/peopleindreamdontsleep/SparkanSpider)

Close

peopleindreamdontsleep / SparkanSpiderView on GitHubMore

• External links
• Readme badge

java爬虫，反爬虫策略、ETL清洗数据，以及spark离线和实时分析新闻并存入ES

☆19Nov 26, 2018Updated 7 years ago

Alternatives and similar repositories for SparkanSpider

Users that are interested in SparkanSpider are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• 0xqq / lighthouse

  View on GitHub
  离线调度, hive, 任务依赖, 任务调度, 大数据开发平台
  ☆14May 10, 2018Updated 8 years ago
• 51bigdata / StockData2Hbase

  View on GitHub
  股票交易数据处理的整个业务流程 数据源--->数据采集--->数据归类--->数据储存--->数据分析--->数据可视化
  ☆31Nov 23, 2016Updated 9 years ago
• lxf44944 / sinaNews_crawler

  View on GitHub
  新浪新闻爬虫
  ☆15Feb 14, 2015Updated 11 years ago
• minminmsn / accesslog-analysis-alarm

  View on GitHub
  日志分析产品，该解决方案整合了filebeat、kafka、logstash、elasticsearch、kibana、grafana、elastalert等开源产品，能够实现海量日志实时分析及错误报警，另外还具有日常报表功能
  ☆23Jan 11, 2019Updated 7 years ago
• lumanjiao / XLS_BigData_Hue

  View on GitHub
  新零售大数据平台-运维监控平台的开发
  ☆14Jan 14, 2019Updated 7 years ago
• Managed hosting for WordPress and PHP on Cloudways • Ad
  Managed hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
• changge458 / userdraw

  View on GitHub
  用户画像代码，根据算法推算出用户的性别和年龄比率
  ☆11Dec 18, 2017Updated 8 years ago
• TavenYin / database-cdc

  View on GitHub
  数据库CDC (Change Data Capture) 核心技术, 持续更新中
  ☆16Apr 30, 2021Updated 5 years ago
• 0xqq / DRDCDeviceMonitor

  View on GitHub
  DRDCDeviceMonitor Flink+Kafka+redis+rabbitmq实现的实时智能运维系统。
  ☆28Aug 30, 2018Updated 8 years ago
• JasonWiki / dw_etl

  View on GitHub
  dw etl 工具 mysql 增量、全量抽取 to hive. 合并 hive 数据表, 等数据平台清洗工具
  ☆10Dec 21, 2016Updated 9 years ago
• kongfy / RandomForest_AdaBoost

  View on GitHub
  随机森林和AdaBoost
  ☆13Oct 20, 2013Updated 13 years ago
• kingsaction / GraphAnalysis

  View on GitHub
  大规模图数据交互式可视化分析平台
  ☆15Apr 23, 2018Updated 8 years ago
• jinshuai86 / Spider

  View on GitHub
  基于Java的多线程爬虫框架
  ☆11Jun 14, 2024Updated 2 years ago
• hxs1216 / Thread

  View on GitHub
  JAVA多线程实用案例：利用CAS解决原子性、实现Callable创建线程（闭锁）、模拟CAS算法、CopyOnWriteArrayList写入并复制、CountDownLatch闭锁、同步锁Lock、多生产多消费、volatile关键字、线程按序交替、Executors线…
  ☆10Jun 26, 2019Updated 7 years ago
• zhoufwind / palo-deploy

  View on GitHub
  使用shell脚本部署Apache Doris (incubating) FE & BE
  ☆11Jul 8, 2019Updated 7 years ago
• AI Agents on DigitalOcean Gradient AI Platform • Ad
  Build production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
• SelfEvoAI / sparkStreamingKafkaPerformance

  View on GitHub
  spark将hdfs数据高性能灌入kafka，然后spark streaming/structured streaming高速消费，关注性能，欢迎提供性能/代码优化建议
  ☆32Mar 24, 2019Updated 7 years ago
• JP6907 / RealTimeLogAnalyze

  View on GitHub
  一个大数据实时流处理日志分析系统 Demo
  ☆30Nov 16, 2022Updated 3 years ago
• gitzh750vp520 / OnlineEdu

  View on GitHub
  视频教育网站
  ☆17Sep 25, 2018Updated 8 years ago
• orangeMask / spider

  View on GitHub
  抖音,淘宝系,常见新闻爬虫
  ☆13Apr 15, 2022Updated 4 years ago
• zm12397 / Analysis-of-web-user-behavior

  View on GitHub
  网站用户行为分析
  ☆15Oct 25, 2018Updated 8 years ago
• CrawlScript / WeiboLoginTool

  View on GitHub
  基于WebCollector的新浪微博爬虫及相关登录工具，如新浪微博Cookie获取
  ☆14Nov 21, 2018Updated 7 years ago
• pyorc / pyorcnews

  View on GitHub
  基于scrapy框架的新闻爬虫
  ☆11Jan 13, 2016Updated 10 years ago
• KeepSorted / PushServer

  View on GitHub
  A high performance push server based on SpringBoot + ConcurrentHashMap + Netty
  ☆16Oct 22, 2019Updated 7 years ago
• digfound / sinacrawler

  View on GitHub
  第一次编写Python网络爬虫，主要使用beautifulsoup4爬取新浪新闻首页新闻列表。成功获取新闻标题、时间、来源、详情、评论数、编辑信息，使用pandas整理数据，并保存到数据库。
  ☆13Dec 7, 2017Updated 8 years ago
• Managed Database hosting by DigitalOcean • Ad
  PostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
• imondo / news-crawler

  View on GitHub
  node 小爬虫，爬取本地新闻
  ☆16May 2, 2024Updated 2 years ago
• madpudding / RelationshipCrawler

  View on GitHub
  知网、万方、专利局爬虫
  ☆11Mar 20, 2019Updated 7 years ago
• Ingram7 / NewsinaSpider

  View on GitHub
  Scrapy 新浪新闻爬虫
  ☆12Aug 26, 2019Updated 7 years ago
• gugug / WeiboNewsProject

  View on GitHub
  系统从互联网爬取资讯，对热点公共事件进行检测、聚合和追踪，多维度分析事件内容，监测时间传播路径，分析用户的观点和和情绪，形成摘要、报告、图表等分析结果，实现公共事件的舆情分析可视化系统，提供专业的舆情监测、分析和预警服务
  ☆104Jul 4, 2018Updated 8 years ago
• goozp / ths-spider-example

  View on GitHub
  完整的 scrapy 爬虫示例，爬取股票和新闻数据
  ☆17Aug 15, 2020Updated 6 years ago
• chmod740 / BaiduBaikeSpider

  View on GitHub
  百度百科多线程爬虫Java源码，数据存储采用了Oracle11g
  ☆13Feb 23, 2017Updated 9 years ago
• ouhamzalhss / ecom-app-frontend

  View on GitHub
  How to build a real e commerce application using new technologies like Angular and Bootstrap.
  ☆19Jul 10, 2021Updated 5 years ago
• apa7 / StockSpider

  View on GitHub
  一个股票爬虫，从和讯、雪球等股票行情网站爬取股票数据, 作为数据分析选股的工具。
  ☆11Jul 25, 2017Updated 9 years ago
• OneGeek / GWT-CompletableFuture

  View on GitHub
  GWT compatible implementation of java.util.concurrent.CompletableFuture and supporting classes
  ☆26Dec 10, 2021Updated 4 years ago
• End-to-end encrypted email - Proton Mail • Ad
  Special offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
• ImYoursQaQ / mashibing-Java-concurrent

  View on GitHub
  Java 多线程
  ☆13Jun 16, 2020Updated 6 years ago
• fullstackyang / weibo-crawler

  View on GitHub
  新型的免登录微博爬虫，自动获取Cookie直接进行抓取和解析微博数据，免去了账号登录的过程，彻底摆脱账号被封的困扰
  ☆37Oct 15, 2017Updated 9 years ago
• free-iot / freeiot-ios

  View on GitHub
  FreeIOT is a open application to interact with multifarious IOT devices.
  ☆10Oct 22, 2015Updated 11 years ago
• water32 / pgAdminIII4GP

  View on GitHub
  解压可执行文件
  ☆11Feb 19, 2021Updated 5 years ago
• dazheng / SparkETL

  View on GitHub
  Implement a complete data warehouse etl using spark SQL
  ☆14Sep 8, 2022Updated 4 years ago
• linzhouzhi / mix-query

  View on GitHub
  基于 spark 混合查询平台，支持不同源数据库的联合查询，mysql hive presto ...
  ☆14Aug 3, 2017Updated 9 years ago
• a123demi / spring-sparkstreaming-kafka-10

  View on GitHub
  spring+spark streaming+kafka 10版本集成和异常问题处理
  ☆17Jul 21, 2017Updated 9 years ago