"奇伢爬虫"是基于sprint boot 、 WebMagic 实现 微信公众号文章、新闻、csdn、info等网站文章爬取，可以动态设置文章爬取规则、清洗规则，基本实现了爬取大部分网站的文章。
☆323Sep 3, 2017Updated 8 years ago
Alternatives and similar repositories for javaCrawling
Users that are interested in javaCrawling are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 微信公众号文章爬虫☆44Sep 1, 2022Updated 3 years ago
- 一个基于webmagic框架二次开发的java爬虫框架实战，已实现能爬取腾讯，搜狐，今日头条（单独集成功能）等资讯内容，配合elasticsearch框架用法，实现了自动爬虫，已投入线上生产使用。☆341Nov 16, 2022Updated 3 years ago
- 基于webmagic + springboot + mybatis的Java爬虫，使用Echarts进行数据可视化分析，提供了从爬虫获取数据到数据持久化、数据可视化分析以及构建简单的代理池等一整套解决方案模板。☆367Oct 26, 2017Updated 8 years ago
- 基于Map/Reduce爬虫,可抽取各大新闻网站的新闻正文并进行分类和聚类☆73Jan 5, 2014Updated 12 years ago
- java爬虫，反爬虫策略、ETL清洗数据，以及spark离线和实时分析新闻并存入ES☆19Nov 26, 2018Updated 7 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 天气爬虫（全国城镇天气自动定时抓取更新，并开放RESTful查询接口），附带代理IP池定时更新并检测其可用性☆367Jun 25, 2018Updated 7 years ago
- 利用Java网络爬虫爬取重庆大学新闻网站数据，依据解析的数据构建的新闻网站☆11Mar 7, 2016Updated 10 years ago
- 用java写的搜狐新闻爬虫☆14May 2, 2017Updated 9 years ago
- 一个简单、敏捷、分布式的支持SpringBoot的Java爬虫框架;An agile, distributed crawler framework.☆1,995Nov 25, 2024Updated last year
- 基于 webmagic 的 Java 爬虫应用☆2,779Jan 8, 2022Updated 4 years ago
- 新浪新闻爬虫☆15Feb 14, 2015Updated 11 years ago
- Easy to use lightweight web crawler（易用的轻量化网络爬虫）☆2,513Jan 23, 2026Updated 3 months ago
- 抓取下载在线视频网站，支持优酷，爱奇艺、Youtube、乐视等☆12Jul 12, 2017Updated 8 years ago
- zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目☆919Apr 2, 2019Updated 7 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Lianjia house spider链家二手房爬虫~ Springboot + Webmagic + Mysql + Redis☆27Apr 22, 2021Updated 5 years ago
- 今日头条科技新闻接口爬虫☆17Sep 26, 2017Updated 8 years ago
- 豆瓣爬虫 爬取热门标签、图书信息、图书评论 系统架构 Webmagic+SSM+Redis+Mysql+ActiveMQ+Druid☆45Apr 24, 2019Updated 7 years ago
- 百科名医的数据爬虫，科室、疾病、症状、检查等类型，包括医疗百科。☆11Dec 21, 2017Updated 8 years ago
- A configurable web spider with a easy-to-use web console☆999Aug 21, 2018Updated 7 years ago
- Spark混合推荐系统大数据监控平台☆11May 1, 2018Updated 8 years ago
- 项目中常用的基础功能实现类。如图片处理、文件操作、短信发送、地图定位、距离计算、MD5、二维码生成与解析、在线支付、消息推送、JSON处理等等。☆11Jan 4, 2016Updated 10 years ago
- SpringBoot+Solr + webmagic JD商品爬取数据，放入solr中做搜索，学习下solr使用☆44Aug 31, 2017Updated 8 years ago
- A scalable web crawler framework for Java.☆11,681Dec 20, 2025Updated 5 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 豆瓣电影爬虫——a crawler which is able to crawl movie detail and short comments, save them to database mysql, also include Sentiment analysis ba…☆70Mar 24, 2019Updated 7 years ago
- spring整合webmagic，mybatis，dungproxy☆29Jun 14, 2023Updated 2 years ago
- 基于WebCollector的新浪微博爬虫及相关登录工具，如新浪微博Cookie获取☆14Nov 21, 2018Updated 7 years ago
- 基于Spring Boot实现的一个简易的Java社区☆898Jun 17, 2022Updated 3 years ago
- 实现视音频的采集，编码，rtmp推送☆15Dec 22, 2015Updated 10 years ago
- 前后分离的通用后台管理系统，前端基于Ant Design Pro-Vue，后端基于SpringBoo 2.x。包含：定时任务管控、基于JWT的配合Shiro的鉴权、一键生成前后端增删改查代码、基于RBAC权限管控、用户管理、组织架构、各类日志(操作日志/定时任务日志/ 登录日…☆14Dec 10, 2022Updated 3 years ago
- springboot+vue页面 REST代码生成器，生成单表的增删改查☆30May 14, 2023Updated 3 years ago
- 使用WebMagic抓取招聘信息，并且持久化到Mysql的例子。☆225Nov 22, 2016Updated 9 years ago
- 一个SpringMVC4+EasyUI的后台管理系统，已投入生产线上使用。下载导入SQL脚本，开箱即用，五分钟完成部署。☆148Dec 16, 2022Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- 多线程爬虫--抓取淘宝商品详情页URL☆129Dec 26, 2018Updated 7 years ago
- 基于微服务的消防物联网云平台☆23Nov 16, 2022Updated 3 years ago
- micro-mall项目是一套电商系统，包括前台商城系统及后台管理系统，基于Alibaba Spring Cloud + MyBatis实现。前台商城系统包含首页门户、商品推荐、商品搜索、商品展示、购物车、订单流程、会员中心、客户服务、帮助中心等模块。后台管理系统包含商品管…☆17Jun 21, 2022Updated 3 years ago
- A simple and flexible web crawler framework for java.☆19Apr 22, 2018Updated 8 years ago
- 一个基于ElasticSearch的业务日志记录工具☆10Nov 5, 2018Updated 7 years ago
- JSender 统一多渠道推送消息格式，支持http接口模板推送和批量推送，适用服务告警通知、设备异常通知等场景。☆10Oct 21, 2022Updated 3 years ago
- 微信公众平台(订阅号、服务号、企业号、小程序)、微信开放平台和微信支付 Java SDK☆1,056Nov 15, 2022Updated 3 years ago