Alternatives and similar repositories for crawler

Users that are interested in crawler are comparing it to the libraries listed below

• theanti9 / PyCrawler
  A python web crawler
  ☆213Updated 4 years ago
• realpython / stack-spider
  ☆167Updated 7 years ago
• manning23 / MSpider
  Spider
  ☆347Updated 3 years ago
• shelmesky / crawler
  淘宝爬虫原型，基于gevent
  ☆48Updated 12 years ago
• omengye / spider
  python Movie Info Web Crawler
  ☆95Updated 8 years ago
• RobinDavid / Pytesser
  Python wrapper for the tesseract OCR engine. The module is based on OpenCV
  ☆178Updated 8 years ago
• Greyh4t / ProxyPool
  Crawl and validate proxies from Internet
  ☆78Updated 9 years ago
• writepython / web-crawler
  Python Web Crawler with Selenium and PhantomJS
  ☆19Updated 8 years ago
• Diastro / Zeek
  Python distributed web scrapper and dynamic crawler
  ☆149Updated 8 years ago
• yoyzhou / weibo_scrapy
  WEIBO_SCRAPY is a Multi-Threading SINA WEIBO data extraction Framework in Python.
  ☆155Updated 8 years ago
• qinxuye / cola
  A high-level distributed crawling framework.
  ☆1,506Updated 3 years ago
• eastrd / HighAnonProxyPool
  A proxy pool that scrapes free anonymous proxies and maintains its proxies' availability.
  ☆92Updated 8 years ago
• mekarpeles / captcha-decoder
  An elementary captcha decoder written in python
  ☆155Updated 10 years ago
• cyh24 / multicpu
  Multi-CPU, Multi-Thread. Implemented in Python.
  ☆80Updated 10 years ago
• bcambel / oss.io
  Developers gathering up
  ☆211Updated 8 years ago
• lllcho / CAPTCHA-breaking
  ☆222Updated 9 years ago
• KeithYue / Zhihu_Spider
  Scrapy the Zhihu content and user social network information
  ☆46Updated 11 years ago
• yrq110 / TinyGoogle
  A search web app built by Flask and Google CSE
  ☆182Updated 3 years ago
• kezakez / python-web-crawler
  ☆38Updated 10 years ago
• tpeng / weibosearch
  A distributed Sina Weibo Search spider base on Scrapy and Redis.
  ☆145Updated 12 years ago
• StephinChou / Pythonspider
  一个简单的python爬虫，原生python+BeautifulSoup
  ☆156Updated 6 years ago
• 2shou / PhantomjsFetcher
  A python web fetcher using phantomjs to mock browser
  ☆180Updated 8 years ago
• liuslevis / weiquncrawler
  This is a crawler for Sina Weiqun website(WAP) information, including given Weiqun's posts, replies, users and their follow relation. Wri…
  ☆141Updated 11 years ago
• orf / cyborg
  Python web scraping framework
  ☆312Updated 8 years ago
• chensoul / scrapy-zhihu-github
  scrapy examples for crawling zhihu and github
  ☆223Updated 3 years ago
• meibenjin / GoogleSearchCrawler
  a tool for crawl Google search results
  ☆403Updated 6 years ago
• n0tr00t / Beehive
  Beehive is an open-source vulnerability detection framework based on Beebeeto-framework. Security researcher can use it to find vulnerabi…
  ☆153Updated 10 years ago
• kenshinx / second-spider
  one more spider based on gevent requests pyquery
  ☆53Updated 11 years ago
• leitro / ustchacker.com
  USTC Hackers' Club (Categories interest website using tornado and bootstrap) python web
  ☆92Updated 11 years ago
• phith0n / python-xss-filter
  Based on native Python module HTMLParser purifier of HTML, To Clear all javascript in html
  ☆116Updated 9 years ago