Alternatives and similar repositories for crawler

Users that are interested in crawler are comparing it to the libraries listed below

• theanti9 / PyCrawler
  A python web crawler
  ☆212Updated 3 years ago
• shelmesky / crawler
  淘宝爬虫原型，基于gevent
  ☆49Updated 12 years ago
• realpython / stack-spider
  ☆167Updated 6 years ago
• omengye / spider
  python Movie Info Web Crawler
  ☆89Updated 8 years ago
• yoyzhou / weibo_scrapy
  WEIBO_SCRAPY is a Multi-Threading SINA WEIBO data extraction Framework in Python.
  ☆154Updated 7 years ago
• tpeng / weibosearch
  A distributed Sina Weibo Search spider base on Scrapy and Redis.
  ☆145Updated 12 years ago
• weizetao / spider-roach
  分布式定向抓取集群
  ☆71Updated 7 years ago
• ZhangBohan / fun_crawler
  Crawl some picture for fun
  ☆162Updated 8 years ago
• wangshunping / weibo_spider
  graduate project, a weibo spider to find some interesting information such as "In social network , people tend to be happy or sad."
  ☆273Updated 9 years ago
• 2shou / PhantomjsFetcher
  A python web fetcher using phantomjs to mock browser
  ☆180Updated 7 years ago
• Greyh4t / ProxyPool
  Crawl and validate proxies from Internet
  ☆77Updated 8 years ago
• manning23 / MSpider
  Spider
  ☆347Updated 2 years ago
• lxyu / weibo
  Python sina weibo sdk. More simpler and cleaner than the official one.
  ☆235Updated 5 years ago
• RobinDavid / Pytesser
  Python wrapper for the tesseract OCR engine. The module is based on OpenCV
  ☆177Updated 7 years ago
• eloyz / reddit
  Scrapy (Python Framework) Example using reddit.com
  ☆54Updated 6 years ago
• kenshinx / second-spider
  one more spider based on gevent requests pyquery
  ☆54Updated 10 years ago
• TeamHG-Memex / arachnado
  Web Crawling UI and HTTP API, based on Scrapy and Tornado
  ☆162Updated 2 years ago
• bbrodriges / pholcidae
  Tiny python web crawler
  ☆169Updated 9 years ago
• Andrew-liu / scrapy_example
  This repository store some example to learn scrapy better
  ☆176Updated 4 years ago
• voliveirajr / seleniumcrawler
  An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
  ☆127Updated 6 years ago
• yanshengli / sina_weibo_crawler
  利用urllib2加beautifulsoup爬取新浪微博
  ☆69Updated 9 years ago
• leitro / ustchacker.com
  USTC Hackers' Club (Categories interest website using tornado and bootstrap) python web
  ☆92Updated 10 years ago
• 4rsh / python
  Developments in Python. — Tools, examples, etc.
  ☆67Updated 4 years ago
• xelzmm / proxy_server_crawler
  an awesome public proxy server crawler based on scrapy framework
  ☆95Updated 8 years ago
• lichald / spider
  python related
  ☆48Updated last year
• salamer / Zhihu_Crawler
  a crawler for zhihu
  ☆94Updated 8 years ago
• liuzheng / weibo
  新浪weibo微博抓取，Python3 support
  ☆77Updated 8 years ago
• lllcho / CAPTCHA-breaking
  ☆223Updated 9 years ago
• yrq110 / TinyGoogle
  A search web app built by Flask and Google CSE
  ☆183Updated 2 years ago
• mjhea0 / Scrapy-Samples
  Scrapy examples crawling Craigslist
  ☆199Updated 9 years ago