GuozhongCrawler的是一个无须配置、便于二次开发的爬虫开源框架，它提供简单灵活的API，只需少量代码即可实现一个爬虫。其设计灵感来源于多个爬虫国内外爬虫框架的总结。采用完全模块化的设计，功能覆盖整个爬虫的生命周期(链接提取、页面下载、内容抽取、持久化)，支持多线程抓取，分布式抓取，并支持自动重试，定制执行js、自定义cookie等功能。在处理网站抓取多次后被封IP的问题上，guozhongCrawler采用动态轮换IP机制有效防止IP被封。另外，源码中的注释及Log输出全部采用通俗易懂的中文。让初学者能有更加深刻的理解
☆103Apr 20, 2015Updated 11 years ago
Alternatives and similar repositories for GuozhongCrawler
Users that are interested in GuozhongCrawler are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A simple and flexible web crawler framework for java.☆19Apr 22, 2018Updated 8 years ago
- dubbox 2.8.1版本 在其基础之上进行扩展☆10Dec 16, 2014Updated 11 years ago
- 轻量级的事件驱动和异步框架☆40Sep 1, 2024Updated 2 years ago
- excel 公用导出组件☆13Jan 28, 2016Updated 10 years ago
- 豆瓣电影爬虫——a crawler which is able to crawl movie detail and short comments, save them to database mysql, also include Sentiment analysis ba…☆68Mar 24, 2019Updated 7 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- 利用WebMagic框架进行58同城数据的抓取☆12Oct 13, 2014Updated 11 years ago
- 基于 Java 的免费开源跨平台系统运行监控服务应用及引擎框架。A free, open source, cross-platform system running monitoring service application and engine framework ba…☆17Aug 13, 2019Updated 7 years ago
- Web/FileSystem Crawler Library☆39Sep 15, 2026Updated last week
- dubbo extension☆11Dec 26, 2015Updated 10 years ago
- 阿里云钉钉交流群收集☆13Feb 13, 2019Updated 7 years ago
- 使用java+httpclient+httpcleaner，多线程、分布式爬去电商网站商品信息，数据存储在hbase上，并使用solr对商品建立索引，使用redis队列存储一个共享的url仓库；使用zookeeper对爬虫节点生命周期进行监视等。☆235Nov 6, 2020Updated 5 years ago
- 抓取gank.io网页☆16Jun 20, 2015Updated 11 years ago
- Android Studio自定义的Activity模板☆11Jun 29, 2018Updated 8 years ago
- 每天三分钟的科技新闻聚合阅读☆18May 15, 2018Updated 8 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- 抓取代理ip，保存有效可用的代理ip☆14Aug 22, 2014Updated 12 years ago
- 数学函数，表达式，公式，坐标，长度，圆，圆弧，方程，计算式的Java语言实现。☆12Dec 28, 2016Updated 9 years ago
- 拉勾网数据爬虫☆32Sep 22, 2017Updated 9 years ago
- 百度搜索指数 对标 股票数据，分析相关性，后面研究 搜索数量、热度 与 股票价值、涨跌预测的 数学模型☆16Jan 25, 2021Updated 5 years ago
- 数据平台(DataPlateform)，最初的设计想法是：当今大数据横行，我们也不能落后。所以就想着写一个这样的平台系统。此项目集爬虫、搜索、Hadoop、Dwr推送、Quartz定时任务于一体的平台，其目的是想通过抓取互联网数据，通过大数据推测人或者某一事物的下一行为。C…☆18Jul 31, 2017Updated 9 years ago
- 对netty4.1的中文翻译和注释☆11Jul 15, 2016Updated 10 years ago
- ☆16Dec 10, 2019Updated 6 years ago
- Wechat LuckyMoney Grabber☆10Feb 8, 2016Updated 10 years ago
- 一个支持多级缓存的分布式缓存系统☆21Dec 27, 2017Updated 8 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- 基于爬虫式对FoFa数据进行爬取，无需FoFa高级会员和F币，未登录，普通用户，基础会员，高级会员均可使用。☆11Jan 15, 2022Updated 4 years ago
- A pure Java IPMI and RMCP implementation.☆41Jan 5, 2020Updated 6 years ago
- 采用领域驱动设计（Domain Driven Design、DDD）实现的CMS系统☆11Aug 8, 2022Updated 4 years ago
- 微信指数爬虫☆24Aug 29, 2018Updated 8 years ago
- Utmost是一款基于Java的面向业务配置平台,能够迅速搭建企业级应用,集成了业界优秀的开源框架,内置任务调度,多维权限管理,规则引擎,工作流引擎,系统监控,ETL,报表,打印等企业常用组件.拥有一流的用户体验和丰富的界面展现.☆25Mar 28, 2017Updated 9 years ago
- 可能是全网最方便的水印图床，支持宝塔一键部署、也支持Docker版部署至服务器或本地电脑☆10Jul 16, 2019Updated 7 years ago
- bboss rpc eclipse project.支持丰富的协议栈(http/netty/mina/jms/webservice/rmi/jgroups/restful) 安全高效,可非常方便地将bboss ioc管理的业务组件发布成RPC服务☆15Jul 11, 2026Updated 2 months ago
- opencart2.0 中文包 简化注册 支付宝 适应中国国情☆10Nov 14, 2014Updated 11 years ago
- Kotlin Builtins Extension☆17Jan 4, 2017Updated 9 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Linux下的设备管理器（支持硬件信息检测、设备驱动安装和硬件性能测试）☆10Oct 22, 2013Updated 12 years ago
- 自定义导出excel☆10Oct 29, 2018Updated 7 years ago
- Chromium-based headless browser for java☆29Oct 14, 2016Updated 9 years ago
- Chrome 扩展，为百度搜索添加必应壁纸。☆11Jun 29, 2015Updated 11 years ago
- 识别5184验证码☆79Jan 30, 2016Updated 10 years ago
- Crawler-for-Douban☆16Mar 29, 2017Updated 9 years ago
- java 基于selenium抓取搜狗微信公众号文章☆50Nov 16, 2015Updated 10 years ago