Alternatives and similar repositories for NewsCrawl

Users that are interested in NewsCrawl are comparing it to the libraries listed below

• javabloger / yuqing

  View on GitHub
  思通舆情 是一款开源免费的舆情系统，支持本地化部署。支持对海量的舆情数据进行多维交叉分析和深度挖掘,为用户户提供全面的舆情数据,专业的舆情分析。
  ☆609Jan 4, 2023Updated 3 years ago
• rrrrat / PassCaptcha

  View on GitHub
  帮助爬虫通过点选验证及滑动验证, 验证码识别
  ☆99Sep 14, 2022Updated 3 years ago
• mic1on / fastrpc

  View on GitHub
  这是一个基于 FastAPI 的 浏览器 RPC 服务端
  ☆60Aug 17, 2023Updated 2 years ago
• mouday / spider-admin-pro

  View on GitHub
  spider-admin-pro 一个集爬虫Scrapy+Scrapyd爬虫项目查看 和 爬虫任务定时调度的可视化管理工具，SpiderAdmin的升级版
  ☆612Nov 10, 2024Updated last year
• lixi5338619 / magical_spider

  View on GitHub
  神奇的蜘蛛🕷，一个几乎适用于所有web端站点的采集方案
  ☆352Aug 23, 2022Updated 3 years ago
• Boris-code / feapder

  View on GitHub
  🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬…
  ☆3,615Dec 19, 2025Updated last month
• Python3Spiders / AllNewsSpider

  View on GitHub
  澎湃新闻，新浪新闻，腾讯新闻，搜狐新闻，新闻联播，泰晤士报，纽约时报，BBCNews，旨在爬取所有新闻门户网站的新闻，禁止将所得数据商用！
  ☆433Oct 18, 2022Updated 3 years ago
• GeneralNewsExtractor / GeneralNewsExtractor

  View on GitHub
  新闻网页正文通用抽取器 Beta 版.
  ☆3,770May 22, 2025Updated 8 months ago
• JSREI / crawler-js-hook-framework-public

  View on GitHub
  JS逆向Hook工具集，开源部分工具到这里
  ☆379Jul 30, 2023Updated 2 years ago
• casual-silva / ppyolo-captcha

  View on GitHub
  使用百度开源ppyolo3目标检测模型识别滑动验证码 极验滑块验证码 识别成功率99% 🎯
  ☆87Oct 21, 2021Updated 4 years ago
• wkunzhi / SpiderUtilPackage

  View on GitHub
  📦 原创开发的 爬虫实用工具 【特定代理池】【特定cookies池】【注册辅助工具】
  ☆118Oct 4, 2019Updated 6 years ago
• wnma3mz / wechat_articles_spider

  View on GitHub
  微信公众号文章的爬虫
  ☆3,355Apr 18, 2024Updated last year
• striver-ing / wechat-spider

  View on GitHub
  开源微信爬虫：爬取公众号所有 文章、阅读量、点赞量和评论内容。易部署。持续维护！！！
  ☆2,752Mar 31, 2023Updated 2 years ago
• WenRichard / Event_Identification_NER_Classfication

  View on GitHub
  互联网舆情企业风险事件的识别和预警，将公司名称进行实体提取，对新闻进行舆情分类，比赛地址为：http://ailab.aiwin.org.cn/competitions/48#learn_the_details
  ☆19May 16, 2021Updated 4 years ago
• ylw00 / cookies_pool_v2

  View on GitHub
  cookie维护
  ☆20Jan 12, 2025Updated last year
• BestToYou / bestV8_release

  View on GitHub
  一个可以跑js的逆向工具。
  ☆580Jun 28, 2023Updated 2 years ago
• stay-leave / weibo-public-opinion-analysis

  View on GitHub
  基于微博数据的舆情分析项目，包括微博爬虫、LDA主题分析和情感分析。
  ☆997Dec 8, 2024Updated last year
• ssssssss-team / spider-flow

  View on GitHub
  新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。
  ☆11,254Jun 14, 2023Updated 2 years ago
• DropsDevopsOrg / ECommerceCrawlers

  View on GitHub
  实战🐍多种网站、电商数据爬虫🕷。包含🕸：淘宝商品、微信公众号、大众点评、企查查、招聘网站、闲鱼、阿里任务、博客园、微博、百度贴吧、豆瓣电影、包图网、全景网、豆瓣音乐、某省药监局、搜狐新闻、机器学习文本采集、fofa资产采集、汽车之家、国家统计局、百度关键词收录数、蜘蛛…
  ☆5,401May 22, 2024Updated last year
• my8100 / scrapydweb

  View on GitHub
  Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI.…
  ☆3,407Feb 19, 2025Updated 11 months ago
• Boris-code / feaplat

  View on GitHub
  爬虫管理系统，支持集群，弹性伸缩。支持运行feapder、scrapy、selenium、playwright等各种框架及脚本
  ☆130Dec 2, 2024Updated last year
• lumina37 / rotate-captcha-crack

  View on GitHub
  基于CNN的旋转验证码解决方案✨Predict the rotation angle of given captcha picture through CNN
  ☆469Jan 17, 2026Updated last month
• bouxin / company-crawler

  View on GitHub
  天眼查爬虫&企查查爬虫，指定关键字爬取公司信息
  ☆698Feb 16, 2023Updated 3 years ago
• jungepro / NetYuQin

  View on GitHub
  网络舆情监测系统
  ☆15Aug 11, 2024Updated last year
• skygongque / Spider

  View on GitHub
  爬虫合集
  ☆127Jan 12, 2024Updated 2 years ago
• easygoingbl / auditlimit

  View on GitHub
  内容审核及速率限制服务
  ☆26May 18, 2025Updated 8 months ago
• firerpa / lamda

  View on GitHub
  The most powerful Android RPA agent framework, next generation of mobile automation robots.
  ☆7,609Dec 13, 2025Updated 2 months ago
• Alpha-su / dbpolicy_crawl

  View on GitHub
  一个新闻政策类爬虫项目，实现上万网站的实时监控、爬取、过滤、存储，具有高可用性和可扩展性。
  ☆40Oct 12, 2022Updated 3 years ago
• wangluozhe / pybestV8

  View on GitHub
  python的bestV8库
  ☆29May 6, 2023Updated 2 years ago
• chencchen / webcrawler

  View on GitHub
  逆向
  ☆404Oct 13, 2022Updated 3 years ago
• kgspider / crawler

  View on GitHub
  K 哥爬虫代码分享，JS 逆向，爬虫进阶。关注公众号：K哥爬虫
  ☆1,334May 12, 2025Updated 9 months ago
• TurboWay / spiderman

  View on GitHub
  基于 scrapy-redis 的通用分布式爬虫框架
  ☆619Mar 31, 2023Updated 2 years ago
• L018 / Trace

  View on GitHub
  A Chrome extension for web analytics
  ☆199Apr 6, 2022Updated 3 years ago
• ReaJason / xhs

  View on GitHub
  基于小红书 Web 端进行的请求封装。https://reajason.github.io/xhs/
  ☆2,011Jul 1, 2025Updated 7 months ago
• sml2h3 / ast_tools

  View on GitHub
  ast基础框架-基于babel
  ☆597Mar 11, 2022Updated 3 years ago
• fanchangrui / catvm

  View on GitHub
  js逆向环境框架基础（持续补充）
  ☆247Sep 5, 2022Updated 3 years ago
• crawlaio / scrapy-redis-sentinel

  View on GitHub
  scrapy-redis-sentinel 基于 scrapy-redis 的基础上 新增 哨兵（sentinel）连接模式 以及 集群（cluster）连接模式。
  ☆30Mar 31, 2023Updated 2 years ago
• ConlinH / aio-scrapy

  View on GitHub
  Implement scrapy with asyncio
  ☆72Sep 4, 2025Updated 5 months ago
• synodriver / pycurl

  View on GitHub
  PycURL - Python interface to libcurl
  ☆47Oct 13, 2023Updated 2 years ago