ddviplinux/crawler-framework external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

ddviplinux/crawler-framework

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/ddviplinux/crawler-framework)

Close

ddviplinux / crawler-frameworkView on GitHubMore

• External links
• Readme badge

分布式爬虫框架,基于webdrvier模拟用户请求,kafka消息传递,分布式网页存储使用hbase,task异步任务多线程解析,提供基础服务如:proxy ip服务和号码验证服务等, proxy page使用H5和we版进行接入

☆13Dec 18, 2015Updated 10 years ago

Alternatives and similar repositories for crawler-framework

Users that are interested in crawler-framework are comparing it to the libraries listed below

• hellokuku / xiaohuangji

  View on GitHub
  人人网小黄鸡
  ☆21Jan 4, 2013Updated 13 years ago
• stephenluu / proxyIpCrawler

  View on GitHub
  抓取代理ip，保存有效可用的代理ip
  ☆13Aug 22, 2014Updated 11 years ago
• multiversx / mx-chain-core-go

  View on GitHub
  mx-chain-go common packages and high level definitions
  ☆12Mar 12, 2026Updated last week
• django-oscar / django-oscar-demo

  View on GitHub
  Django Oscar demo site
  ☆13Aug 12, 2016Updated 9 years ago
• xtuhcy / webkit4j

  View on GitHub
  Chromium-based headless browser for java
  ☆28Oct 14, 2016Updated 9 years ago
• yyd19921214 / Heze

  View on GitHub
  一个模仿Kafka的简单消息中间件
  ☆14Jun 29, 2022Updated 3 years ago
• JackonYang / distributed-vertical-crawlers

  View on GitHub
  分布式垂直爬虫框架 & 爬虫们
  ☆15Aug 22, 2015Updated 10 years ago
• aptivate / cmsbootstrap

  View on GitHub
  Django-CMS basic theme with Bootstrap to get you started quickly. Low bandwidth and mobile friendly.
  ☆18Nov 16, 2017Updated 8 years ago
• guangruixiao / vue-demo

  View on GitHub
  ☆13May 2, 2017Updated 8 years ago
• liushuishang / YayCrawler

  View on GitHub
  基于java的分布式爬虫框架
  ☆10Dec 16, 2022Updated 3 years ago
• xiaoming198 / apks

  View on GitHub
  ☆12Dec 10, 2018Updated 7 years ago
• zhongxunking / sync

  View on GitHub
  Sync是一款分布式场景下基于Redis的安全高效的线程同步组件，提供分布式可重入互斥锁、分布式可重入读写锁、分布式信号量。提供相应注解，使用简单，可与spring-boot无缝集成。
  ☆13Oct 8, 2022Updated 3 years ago
• scrapy-plugins / scrapy-streaming

  View on GitHub
  ☆19Oct 12, 2016Updated 9 years ago
• luodw / MyDB

  View on GitHub
  用C++实现的一个简单NoSQL
  ☆15Apr 10, 2016Updated 9 years ago
• shiyanlou / library

  View on GitHub
  shiyanlou public library
  ☆15Apr 24, 2020Updated 5 years ago
• Idun / tavern-card-crafter-v3

  View on GitHub
  使用AI编程创建的 SillyTavern 角色卡制作工具
  ☆18Jun 16, 2025Updated 9 months ago
• opendistro-for-elasticsearch / asynchronous-search

  View on GitHub
  Asynchronous search makes it possible for users to run queries in the background, allowing users to track the progress, and retrieve par…
  ☆23Apr 21, 2021Updated 4 years ago
• G5t4r / WeProtocol

  View on GitHub
  微信ipad协议，五端登录版本
  ☆23Aug 15, 2025Updated 7 months ago
• shenbaise / goodcrawler

  View on GitHub
  网络爬虫
  ☆51Mar 18, 2014Updated 12 years ago
• WildDogTeam / video-demo-web-combined

  View on GitHub
  视频会议+白板+课件共享+IM
  ☆13Nov 10, 2017Updated 8 years ago
• tbwork / alipay_edit_typer

  View on GitHub
  Just a DEMO to demonstrate how to use JNA to type chars into alipay's password edit control automatically.
  ☆12Dec 21, 2017Updated 8 years ago
• android-cjj / createcjj

  View on GitHub
  ☆22Sep 14, 2014Updated 11 years ago
• Mr-LuXiaoHua / dohko-state-machine

  View on GitHub
  简单状态机实现。同时以简化的订单状态机为例子进行了说明。
  ☆15Oct 13, 2020Updated 5 years ago
• systemt1st / SiliconFlowregister

  View on GitHub
  硅基流动注册机
  ☆15Mar 28, 2025Updated 11 months ago
• mjkillough / iis-asgi-handler

  View on GitHub
  Module to allow IIS to act as an ASGI Interface Server for Django Channels. (HTTP and WebSockets)
  ☆15Nov 14, 2016Updated 9 years ago
• penghaiping / selling-partner-sdk

  View on GitHub
  Amazon Selling Partner JAVA SDK SP API
  ☆15Jul 5, 2021Updated 4 years ago
• yinheli / mirror_mygithub

  View on GitHub
  把 github 的项目备份到本地
  ☆14Dec 30, 2015Updated 10 years ago
• OrPolyzos / spring-web-initializr-plugin

  View on GitHub
  Swip - Plugin for IntelliJ IDEA that can create a fully functional (Spring Boot) WebApp with just a few clicks
  ☆13Jan 4, 2020Updated 6 years ago
• gabrielpreda / generative_ai

  View on GitHub
  Kaggle Notebooks, Utility Scripts using Generative AI tools to check new models, fine tune models, test with various prompts, create Retr…
  ☆17Mar 8, 2026Updated last week
• CL0610 / effective-java-learning

  View on GitHub
  ☆15Jul 26, 2018Updated 7 years ago
• simplebam / ReadHub

  View on GitHub
  每天三分钟的科技新闻聚合阅读
  ☆18May 15, 2018Updated 7 years ago
• JS-Hao / FFAnalysis

  View on GitHub
  闲鱼数据分析小助手，定期抓取商品信息，分析价格走势
  ☆18Sep 12, 2018Updated 7 years ago
• Shkin1 / hathor

  View on GitHub
  SQL语法词法分析 SQL表级血缘 SQL字段级别血缘 SQL函数血缘 SQL编译器
  ☆17Nov 1, 2022Updated 3 years ago
• tiepologian / dbpediaNeo4j

  View on GitHub
  Python script for importing DBpedia nodes and relationships into Neo4j
  ☆14Mar 15, 2014Updated 12 years ago
• sunpeak / Mind-Map

  View on GitHub
  各种安全相关思维导图整理收集
  ☆11Sep 7, 2015Updated 10 years ago
• httaotao / EsSourceStudy

  View on GitHub
  这个项目是《Elasticsearchsearch源码解析与优化实战》一书的笔记，目前正在整理当中
  ☆27Jul 9, 2020Updated 5 years ago
• ring04h / ChromeDebug

  View on GitHub
  Chrome DevTools Network debuger Examples
  ☆24Nov 24, 2016Updated 9 years ago
• begmaroman / acme-dns-route53

  View on GitHub
  Tool to obtain certs from Let's Encrypt using DNS-01 challenge with Route53 and Amazon Certificate Manager
  ☆25May 21, 2019Updated 6 years ago
• derekparker / Splache

  View on GitHub
  Web server written in c++
  ☆18Jan 14, 2013Updated 13 years ago