Germey/AwesomeWebScraping external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

Germey/AwesomeWebScraping

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/Germey/AwesomeWebScraping)

Close

Germey / AwesomeWebScrapingView on GitHubMore

• External links
• Readme badge

List of libraries, tools and APIs for web scraping and data processing.

☆258Mar 12, 2026Updated last week

Alternatives and similar repositories for AwesomeWebScraping

Users that are interested in AwesomeWebScraping are comparing it to the libraries listed below

• MuggleK / Decrypt_JS

  View on GitHub
  包含一些网站的登陆参数的解密和部分网址的参数解密，并结合Node.JS和Python进行破解。
  ☆18Sep 3, 2021Updated 4 years ago
• luzihang123 / scrapyd_pyppeteer

  View on GitHub
  可以用于scrapydweb的scrapyd节点，使用pyppeteer，在scrapy中异步使用
  ☆12Dec 8, 2022Updated 3 years ago
• jiyulany / Fchrome

  View on GitHub
  这是一款对chromium源码进行定制的浏览器,支持爬虫/JS逆向工程师进行辅助分析网页
  ☆339Dec 25, 2024Updated last year
• Python3WebSpider / ScrapyPyppeteer

  View on GitHub
  Scrapy Pyppeteer Demo
  ☆12Jul 30, 2020Updated 5 years ago
• ylw00 / qxVm

  View on GitHub
  qxVm补环境框架(纯js实现)
  ☆487Nov 25, 2024Updated last year
• yint-tech / malenia-open

  View on GitHub
  malenia是一个对代理IP池进行二次组合的工具，用户可以将多个采购的IP资源进行二次组合，使用统一的账户密码，服务器端口等信息进行统一访问。
  ☆13Jan 2, 2026Updated 2 months ago
• kingname / SifouSource

  View on GitHub
  Python 业务开发常见错误案例集 配套源代码
  ☆10Dec 19, 2020Updated 5 years ago
• Sruiis / fix_broswer_enviroment_frame

  View on GitHub
  自用补环境框架
  ☆21Jun 23, 2023Updated 2 years ago
• wangluozhe / pybestV8

  View on GitHub
  python的bestV8库
  ☆29May 6, 2023Updated 2 years ago
• lorien / awesome-web-scraping

  View on GitHub
  List of libraries, tools and APIs for web scraping and data processing.
  ☆7,807Mar 4, 2026Updated 2 weeks ago
• sml2h3 / ast_tools

  View on GitHub
  ast基础框架-基于babel
  ☆598Mar 11, 2022Updated 4 years ago
• wangluozhe / requests

  View on GitHub
  用于快速请求HTTP或HTTPS，并支持修改ja3、ja4指纹
  ☆560Jan 10, 2026Updated 2 months ago
• Germey / hooker

  View on GitHub
  🔥🔥hooker是一个基于frida实现的逆向工具包。为逆向开发人员提供统一化的脚本包管理方式、通杀脚本、自动化生成hook脚本、内存漫游探测activity和service、firda版JustTrustMe
  ☆10Jun 3, 2021Updated 4 years ago
• Gerapy / GerapyPlaywright

  View on GitHub
  Downloader Middleware to support Playwright in Scrapy & Gerapy
  ☆111Mar 6, 2022Updated 4 years ago
• BestToYou / bestV8_release

  View on GitHub
  一个可以跑js的逆向工具。
  ☆580Jun 28, 2023Updated 2 years ago
• bnmgh1 / NodeSandbox

  View on GitHub
  Node补环境框架
  ☆447Jan 28, 2025Updated last year
• crawlaio / scrapy-redis-sentinel

  View on GitHub
  scrapy-redis-sentinel 基于 scrapy-redis 的基础上 新增 哨兵（sentinel）连接模式 以及 集群（cluster）连接模式。
  ☆30Mar 31, 2023Updated 2 years ago
• Tsaiboss / decodeObfuscator

  View on GitHub
  免安装一键还原Obfuscator混淆过的代码
  ☆50Jun 9, 2024Updated last year
• aioqzone / pychaosvm

  View on GitHub
  A Python envirionment for Tencent ChaosVM.
  ☆97Dec 30, 2025Updated 2 months ago
• Gerapy / Gerapy

  View on GitHub
  Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
  ☆3,502Oct 29, 2024Updated last year
• bnmgh1 / node-sandbox

  View on GitHub
  魔改Node补环境框架
  ☆240Sep 2, 2023Updated 2 years ago
• Gerapy / GerapyAutoExtractor

  View on GitHub
  Auto Extractor Module
  ☆334Aug 19, 2024Updated last year
• downdawn / JSreverse

  View on GitHub
  js逆向和爬虫
  ☆332Jan 12, 2023Updated 3 years ago
• chencchen / webcrawler

  View on GitHub
  逆向
  ☆405Oct 13, 2022Updated 3 years ago
• skygongque / match-yuanrenxue

  View on GitHub
  yuanrenxue 爬虫攻防赛题目练习
  ☆37Feb 15, 2022Updated 4 years ago
• freeFV / fridaUiTools

  View on GitHub
  frida工具的缝合怪
  ☆10Jun 25, 2021Updated 4 years ago
• Germey / Blog

  View on GitHub
  My Personal Blog
  ☆13Mar 8, 2026Updated last week
• qqizai / CrackJs

  View on GitHub
  记录一下js逆向的网站
  ☆233May 22, 2023Updated 2 years ago
• synodriver / awsome_scrapy_utils

  View on GitHub
  awsome scrapy utils
  ☆57Feb 14, 2026Updated last month
• pysunday / rs-reverse

  View on GitHub
  瑞数vmp纯算法逆向，补环境框架sdenv的灵感来源
  ☆524Feb 17, 2026Updated last month
• cilame / v_jstools

  View on GitHub
  模仿着写一个 chrome 插件，用来快速调试前端 js 代码。
  ☆2,927Dec 22, 2025Updated 2 months ago
• pysunday / sdenv

  View on GitHub
  补环境框架sdenv，完美过瑞数vmp理论通杀，配合瑞数vmp纯算法逆向项目（rs-reverse）更好学习原理
  ☆660Dec 22, 2025Updated 2 months ago
• rrrrat / PassCaptcha

  View on GitHub
  帮助爬虫通过点选验证及滑动验证, 验证码识别
  ☆100Sep 14, 2022Updated 3 years ago
• FlorianREGAZ / Python-Tls-Client

  View on GitHub
  Advanced HTTP Library
  ☆816Jul 30, 2024Updated last year
• wangjingyu001 / my_js_parser

  View on GitHub
  ☆24Jan 22, 2025Updated last year
• Vladislao / proxy-supervisor

  View on GitHub
  Refresh, monitor and balance your proxies
  ☆16Jun 9, 2025Updated 9 months ago
• bpcreech / PyMiniRacer

  View on GitHub
  PyMiniRacer is a V8 bridge in Python.
  ☆196Feb 2, 2026Updated last month
• asyncins / spiderbook

  View on GitHub
  《Python3 网络爬虫宝典》随书配套代码
  ☆21Aug 29, 2020Updated 5 years ago
• q-m / scrapyd-k8s

  View on GitHub
  Scrapyd on container infrastructure
  ☆16Apr 11, 2025Updated 11 months ago