腾讯新闻、知乎话题、微博粉丝，Tumblr爬虫、斗鱼弹幕、妹子图爬虫、分布式设计等
☆302Jun 6, 2025Updated last year
Alternatives and similar repositories for awesome_crawl
Users that are interested in awesome_crawl are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 基于scrapy-redis实现分布式爬虫，爬取知乎所有问题及对应的回答，集成selenium模拟登录、英文验证码及倒立文字验证码识别、随机生成User-Agent、IP代理、处理302重定向问题等等☆61Apr 3, 2019Updated 7 years ago
- 今日头条科技新闻接口爬虫☆17Sep 26, 2017Updated 9 years ago
- 最右APP爬虫，用Python爬取最右APP段子数据和视频 弹幕。☆22Jun 29, 2019Updated 7 years ago
- 新闻爬虫，爬取新浪、搜狐、新华网即时财经新闻。☆200May 9, 2020Updated 6 years ago
- 基于scrapy的新闻爬虫☆104Apr 18, 2020Updated 6 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Python分布式爬虫学习笔记，各种Demo同步☆12Aug 21, 2019Updated 7 years ago
- 新闻抓取（微信、微博、头条...）☆225Dec 8, 2022Updated 3 years ago
- Scrapy爬虫实战系列，从零开始爬取腾讯百度淘宝知乎各大网站内容 \n 12306刷票脚本系列☆80Apr 2, 2019Updated 7 years ago
- Weibo's daily TOP5 hotkey. 自动爬取、筛选新浪微博每日热搜词 TOP5。https://github.com/TauWu/weibo_daily_hotkey/blob/master/data/data.md☆36Apr 18, 2021Updated 5 years ago
- 微博爬虫 有问题欢迎提出来☆17Jul 2, 2019Updated 7 years ago
- 《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用n…☆136Jun 26, 2019Updated 7 years ago
- A free API for Google Translate. 免费的谷歌翻译，与谷歌翻译网页版相同，可选国内服务器。亲测一日300万字没问题。☆13Nov 22, 2019Updated 6 years ago
- 微博粉丝情绪分析☆44May 28, 2017Updated 9 years ago
- 使用Scrapy编写的拉勾网爬虫，添加了代理IP池、增量爬取机制☆11May 22, 2023Updated 3 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- 我的博文汇总☆10Sep 3, 2018Updated 8 years ago
- 新闻爬虫 (腾讯,网易,新浪,今日头条,搜狐,凤凰网,腾讯滚动新闻)☆58Jun 6, 2018Updated 8 years ago
- 超高速异步协程Python爬虫☆81Feb 15, 2023Updated 3 years ago
- 汤不热 python 多线程爬虫☆458Jul 22, 2020Updated 6 years ago
- Software Update Server 软件更新服务☆22Jul 9, 2024Updated 2 years ago
- 《精通scrapy网络爬虫》中代码☆10May 15, 2020Updated 6 years ago
- API of DouYin for Humans used to Crawl Popular Videos and Musics☆653Jan 29, 2020Updated 6 years ago
- 📺 B 站全站视频信息爬虫☆703Feb 17, 2019Updated 7 years ago
- 新浪微博爬虫(Sina weibo spider)，百度搜索结果 爬虫☆195Jul 17, 2023Updated 3 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- A simple distributed crawler for zhihu && data analysis☆192Dec 7, 2022Updated 3 years ago
- A distributed crawler for weibo, building with celery and requests.☆4,784Jul 11, 2020Updated 6 years ago
- 🚀🚀文书网cookie获取 2020-08-23 依旧可行。（已终结）☆51Aug 23, 2020Updated 6 years ago
- 自写爬虫爬取知乎问题及回答☆39Jun 10, 2019Updated 7 years ago
- proxy_scrapy是一个scrapy搭建的代理模块 ，主要包括代理抓取、代理测试和使用代理三个模块。包括了对主要的代理网站的抓取和代理稳定性的测试，并整合进scrapy爬虫当中。☆10Jan 20, 2017Updated 9 years ago
- Scrapy 爬虫框架教程源码☆109Aug 23, 2019Updated 7 years ago
- 新浪微博爬虫，功能包括：爬取用户信息、关注、粉丝，爬取超级话题用户及粉丝相关信息☆30Aug 17, 2018Updated 8 years ago
- 微信公众号文章的爬虫☆3,484Apr 18, 2024Updated 2 years ago
- 美国股票爬取（NASDAQ，AMEX，NYSE）☆17Nov 24, 2016Updated 9 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- 拍拍贷"魔镜杯”风控大赛☆12Dec 22, 2016Updated 9 years ago
- 新闻网站爬虫,目前能够爬取网易，新浪，qq，搜狐等三家网站的新闻页面，并保存到本地。☆34Jun 12, 2015Updated 11 years ago
- 抖音,淘宝系,常见新闻爬虫☆13Apr 15, 2022Updated 4 years ago
- 今日头条 、淘宝 、微博 、斗鱼 、抖音 、哔哩哔哩 、有道翻译、steam网站以及网易云音乐爬取☆61Apr 17, 2020Updated 6 years ago
- 🙌Easily download all the videos from TikTok(amemv).下载指定的 抖音（Douyin） 号的视频,抖音爬虫☆2,666Nov 28, 2019Updated 6 years ago
- 图虫网爬虫☆16Jan 2, 2019Updated 7 years ago
- 持续维护的新浪微博采集工具🚀🚀🚀☆4,128Jun 30, 2026Updated 4 months ago