Alternatives and similar repositories for crawler

Users that are interested in crawler are comparing it to the libraries listed below

• keybase / python-triplesec
  A Python port of the triplesec library.
  ☆82Updated 2 years ago
• atupal / ccrawler
  A distrubuted crawler ues celery.
  ☆15Updated 11 years ago
• ssteuteville / scrapyz
  "Scrape Easy" - an extension of the Scrapy framework.
  ☆186Updated 9 years ago
• bbrodriges / pholcidae
  Tiny python web crawler
  ☆169Updated 9 years ago
• collinalexbell / Email-Autoresponder
  ☆21Updated 10 years ago
• Alir3z4 / python-sanitize
  Bringing sanity to world of messed-up data
  ☆66Updated 11 years ago
• scrapinghub / exporters
  Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations
  ☆40Updated last year
• staceysern / bt-twisted
  A BitTorrent client written in Python using Twisted
  ☆33Updated 11 years ago
• niklasb / webkit-server
  [not actively maintained] The C++ webkit-server from capybara-webkit with useful extensions and Python bindings
  ☆48Updated 5 years ago
• orf / simple
  Simple is a clone of Obtvse written in Python running on Flask.
  ☆510Updated 7 years ago
• aaronsw / pytorctl
  ☆56Updated 2 years ago
• papaya-mobile / cyber
  A distributed asynchronous socket framework of Python
  ☆13Updated 9 years ago
• eklitzke / tornado-dns
  Asynchronous DNS lookups for Tornado
  ☆49Updated 10 years ago
• arkaitzj / wsvnc
  In browser VNC client through websockets
  ☆32Updated 14 years ago
• TeamHG-Memex / undercrawler
  A generic crawler
  ☆78Updated 2 weeks ago
• amix / neuralist
  A Python interface to access neural-redis
  ☆79Updated 9 years ago
• marklit / mass-ipv4-whois
  Mass IPv4 WHOIS Collection Tool
  ☆18Updated 9 years ago
• gtoonstra / remap
  MapReduce platform in python
  ☆34Updated 10 years ago
• getdnsapi / getdns-python-bindings
  Python bindings for getdns
  ☆26Updated 5 years ago
• agoragames / kairos
  Python module for time series data in Redis and Mongo
  ☆208Updated 4 years ago
• csirtfoundry / BulkWhois
  Python interfaces to popular bulk WHOIS servers such as Shadowserver and Team Cymru.
  ☆21Updated 14 years ago
• jianxunio / yascrapy
  A high-performance distributed web crawling & scraping framework written with golang and python.
  ☆30Updated 9 years ago
• mehmetkose / tangrowth
  Python3.5 async crawler example with aiohttp and asyncio
  ☆147Updated 8 years ago
• andrewjw / celery-crawler
  A Django based search engine powered by CouchDB, celery and whoosh.
  ☆48Updated 10 years ago
• xmikos / pyxolotl
  Send and receive messages encrypted with Axolotl (Double Ratchet) protocol
  ☆45Updated 7 years ago
• marklit / whois
  Collects WHOIS details for every IPv4 netblock. Reports supported via Elasticsearch.
  ☆103Updated 7 years ago
• yspanchal / docli
  Digital Ocean Command Line Interface
  ☆33Updated 2 years ago
• hawkowl / rproxy
  A HTTP reverse proxy with TLS+SNI support.
  ☆40Updated 6 years ago
• dinever / antispam
  Dead simple bayesian anti-spam classifier written in Python.
  ☆100Updated 7 years ago
• scrapinghub / scrapylib
  Collection of Scrapy utilities (extensions, middlewares, pipelines, etc)
  ☆33Updated 7 years ago