Alternatives and similar repositories for crawler

Users that are interested in crawler are comparing it to the libraries listed below

• twisted / vertex
  Vertex is a generalized, secure, peer-to-peer communications platform; formerly of the divmod.org project (https://code.launchpad.net/div…
  ☆74Updated 6 years ago
• eklitzke / tornado-dns
  Asynchronous DNS lookups for Tornado
  ☆49Updated 10 years ago
• atupal / ccrawler
  A distrubuted crawler ues celery.
  ☆16Updated 11 years ago
• andrewjw / celery-crawler
  A Django based search engine powered by CouchDB, celery and whoosh.
  ☆49Updated 9 years ago
• staceysern / bt-twisted
  A BitTorrent client written in Python using Twisted
  ☆33Updated 11 years ago
• TeamHG-Memex / scrapy-dockerhub
  [UNMAINTAINED] Deploy, run and monitor your Scrapy spiders.
  ☆11Updated 10 years ago
• bbrodriges / pholcidae
  Tiny python web crawler
  ☆169Updated 9 years ago
• mitmproxy / netlib
  🗃️ This repository has been integrated into mitmproxy/mitmproxy.
  ☆66Updated 9 years ago
• papaya-mobile / cyber
  A distributed asynchronous socket framework of Python
  ☆13Updated 9 years ago
• dinever / antispam
  Dead simple bayesian anti-spam classifier written in Python.
  ☆101Updated 7 years ago
• orf / simple
  Simple is a clone of Obtvse written in Python running on Flask.
  ☆510Updated 6 years ago
• ssteuteville / scrapyz
  "Scrape Easy" - an extension of the Scrapy framework.
  ☆186Updated 9 years ago
• collinalexbell / Email-Autoresponder
  ☆21Updated 10 years ago
• keybase / python-triplesec
  A Python port of the triplesec library.
  ☆82Updated last year
• kakwa / pygraph_redis
  Simple python library to manipulate directed graphs in redis
  ☆23Updated 8 years ago
• amix / neuralist
  A Python interface to access neural-redis
  ☆79Updated 9 years ago
• fiorix / txtraceroute
  An asynchronous, pure python, traceroute with geolocation information
  ☆44Updated 10 years ago
• TeamHG-Memex / undercrawler
  A generic crawler
  ☆78Updated 7 years ago
• vaulstein / 7daysOfCode
  #7DaysOfCode Challenge. Next step #100DaysOfCode
  ☆11Updated 8 years ago
• aaronsw / pytorctl
  ☆56Updated last year
• gtoonstra / remap
  MapReduce platform in python
  ☆34Updated 10 years ago
• sasha42 / Mailchimp-utility
  Mailchimp signup utility based on flask + aws lambda/s3
  ☆12Updated 9 years ago
• avelino / bottle-auth
  Bootle authentication, for Personal, Google, Twitter and facebook
  ☆29Updated 8 years ago
• agoragames / kairos
  Python module for time series data in Redis and Mongo
  ☆208Updated 4 years ago
• getdnsapi / getdns-python-bindings
  Python bindings for getdns
  ☆25Updated 4 years ago
• niklasb / webkit-server
  [not actively maintained] The C++ webkit-server from capybara-webkit with useful extensions and Python bindings
  ☆48Updated 4 years ago
• Alir3z4 / python-sanitize
  Bringing sanity to world of messed-up data
  ☆66Updated 11 years ago
• atl / thrifty-p2p
  A very simple python peer-to-peer implementation using consistent hashing (and an example distributed key-value store layered atop it) us…
  ☆30Updated 16 years ago
• scrapinghub / exporters
  Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations
  ☆40Updated last year
• linsomniac / nanomon
  Extremely lightweight monitoring system.
  ☆26Updated 10 years ago