TurboWay/spiderman external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

TurboWay/spiderman

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/TurboWay/spiderman)

Close

TurboWay / spidermanView on GitHubMore

• External links
• Readme badge

基于 scrapy-redis 的通用分布式爬虫框架

☆616Mar 31, 2023Updated 3 years ago

Alternatives and similar repositories for spiderman

Users that are interested in spiderman are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• CasterWx / kibana-eye

  View on GitHub
  ELK数据报表定时任务管理平台，定时任务统计上报应用性能指标(基于时间轮)
  ☆24Oct 9, 2020Updated 5 years ago
• mouday / spider-admin-pro

  View on GitHub
  spider-admin-pro 一个集爬虫Scrapy+Scrapyd爬虫项目查看 和 爬虫任务定时调度的可视化管理工具，SpiderAdmin的升级版
  ☆610May 29, 2026Updated last week
• Boris-code / boris-spider

  View on GitHub
  boris-spider是一款使用Python语言编写的爬虫框架，于多年的爬虫业务中不断磨合而诞生，相比于scrapy，该框架更易上手，且又满足复杂的需求，支持分布式及批次采集。
  ☆85Jan 21, 2022Updated 4 years ago
• shengchenyang / AyugeSpiderTools

  View on GitHub
  使 scrapy 开发不用在意 item，pipeline，middleware 等通用场景下模块的编写，解放开发者的双手。
  ☆100Updated this week
• crawlab-team / crawlab

  View on GitHub
  Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
  ☆12,209Feb 10, 2026Updated 4 months ago
• Serverless GPU API endpoints on Runpod - Get Bonus Credits • Ad
  Skip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
• TurboWay / big_screen

  View on GitHub
  数据大屏可视化
  ☆1,866Jan 22, 2026Updated 4 months ago
• ydf0509 / boost_spider

  View on GitHub
  用户自由无束缚的分布式光速python爬虫函数执行框架,写法和性能远远暴击仿scrapy api式框架
  ☆25Mar 16, 2026Updated 2 months ago
• polyrabbit / WeCron

  View on GitHub
  微信上的定时提醒 - Cron on WeChat
  ☆689Mar 16, 2026Updated 2 months ago
• my8100 / scrapydweb

  View on GitHub
  Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI.…
  ☆3,405Feb 19, 2025Updated last year
• lixi5338619 / weixin-spider

  View on GitHub
  《微信公众号采集系统》微信公众号文章的阅读数、在看数、评论数、评论列表，还有微信公众号的账号基本信息。
  ☆186Apr 29, 2022Updated 4 years ago
• my8100 / files

  View on GitHub
  Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects
  ☆418Mar 1, 2025Updated last year
• Boris-code / feapder

  View on GitHub
  🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬…
  ☆3,705Jun 1, 2026Updated last week
• TurboWay / bigdata_practice

  View on GitHub
  大数据分析可视化实践
  ☆278Nov 4, 2020Updated 5 years ago
• parzulpan / livebox

  View on GitHub
  A cross-platform network media aggregation application
  ☆1,065May 22, 2023Updated 3 years ago
• Wordpress hosting with auto-scaling - Free Trial Offer • Ad
  Fully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
• TurboWay / pybigdata

  View on GitHub
  使用 python 操作大数据的各种组件
  ☆68Feb 17, 2023Updated 3 years ago
• Gerapy / Gerapy

  View on GitHub
  Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
  ☆3,505Oct 29, 2024Updated last year
• wkunzhi / Python3-Spider

  View on GitHub
  Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️
  ☆3,363Nov 3, 2023Updated 2 years ago
• OgrBear / Spider-Crack_Js

  View on GitHub
  爬虫js解密、python解密 大众点评|中国移动|新浪微博|汽车之家|Steam|中华英才网|拼多多|36氪|今日头条... 欢迎Star
  ☆345Dec 31, 2020Updated 5 years ago
• jwenjian / artline-demo

  View on GitHub
  Runnable Python demo of ArtLine
  ☆133Jan 4, 2021Updated 5 years ago
• fat-garage / how-to-get-rich-without-getting-lucky

  View on GitHub
  ☆947Oct 10, 2020Updated 5 years ago
• AndrewAndrea / spider_demo

  View on GitHub
  个人爬虫、逆向练习代码。文书web、app；facebook；毒；省市地区数据抓取，qcc 类似ob混淆, tb 滑动 demo，头条_signature等参数
  ☆54Feb 20, 2021Updated 5 years ago
• DuanJiaNing / GoMailer

  View on GitHub
  轻量电子邮件推送服务(A lightly email sending service for Go)
  ☆90Apr 3, 2020Updated 6 years ago
• openspug / spug

  View on GitHub
  开源运维平台：面向中小型企业设计的轻量级无Agent的自动化运维平台，整合了主机管理、主机批量执行、主机在线终端、文件在线上传下载、应用发布部署、在线任务计划、配置中心、监控、报警等一系列功能。
  ☆11,004May 14, 2026Updated 3 weeks ago
• Managed hosting for WordPress and PHP on Cloudways • Ad
  Managed hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
• leolovenet / qqwry2mmdb

  View on GitHub
  为 Wireshark 能使用纯真网络 IP 数据库(QQwry)而提供的格式转换工具
  ☆170Oct 1, 2024Updated last year
• lixi5338619 / magical_spider

  View on GitHub
  神奇的蜘蛛🕷，一个几乎适用于所有web端站点的采集方案
  ☆347Aug 23, 2022Updated 3 years ago
• rmax / scrapy-redis

  View on GitHub
  Redis-based components for Scrapy.
  ☆5,637May 19, 2026Updated 3 weeks ago
• victorqribeiro / fingerPool

  View on GitHub
  A pool game in JavaScript using my sphereCollision project
  ☆116Feb 27, 2021Updated 5 years ago
• HegemonyTao / crawlProject

  View on GitHub
  今日头条 、淘宝 、微博 、斗鱼 、抖音 、哔哩哔哩 、有道翻译、steam网站以及网易云音乐爬取
  ☆60Apr 17, 2020Updated 6 years ago
• poozhu / crawler-for-github-trending

  View on GitHub
  🕷️ A node crawler for github trending.
  ☆185Nov 17, 2025Updated 6 months ago
• DingZaiHub / PythonSpider

  View on GitHub
  JS逆向系列教程，模拟登录，AES、RSA、DES加密等，持续更新，欢迎 star！
  ☆434Apr 11, 2021Updated 5 years ago
• kingking888 / CommNewsExtractor

  View on GitHub
  通用文章提取，正文，标题，时间，作者，图片，音视频，联系方式等
  ☆23Mar 19, 2023Updated 3 years ago
• cxapython / discogs_aio_spider

  View on GitHub
  基于httpx的一个大型项目 ，爬取黑胶唱片网站 Discogs
  ☆102Jul 14, 2025Updated 10 months ago
• Deploy to Railway using AI coding agents - Free Credits Offer • Ad
  Use Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
• mic1on / fastrpc

  View on GitHub
  这是一个基于 FastAPI 的 浏览器 RPC 服务端
  ☆59Aug 17, 2023Updated 2 years ago
• rrrrat / PassCaptcha

  View on GitHub
  帮助爬虫通过点选验证及滑动验证, 验证码识别
  ☆99Sep 14, 2022Updated 3 years ago
• guoxw / yaojianjuruishu

  View on GitHub
  药监局瑞数反爬学习
  ☆51Dec 2, 2020Updated 5 years ago
• eryajf / magic-of-sysuse-scripts

  View on GitHub
  🪄 运维外挂小工具
  ☆293Oct 9, 2021Updated 4 years ago
• qqizai / CrackApp

  View on GitHub
  安卓逆向相关
  ☆29Jan 2, 2021Updated 5 years ago
• jhao104 / proxy_pool

  View on GitHub
  Python ProxyPool for web spider
  ☆23,384Jun 3, 2026Updated last week
• apioak / apioak

  View on GitHub
  Full Lifecycle Management API Gateway.
  ☆414Jan 26, 2026Updated 4 months ago