Alternatives and similar repositories for crawler

Users that are interested in crawler are comparing it to the libraries listed below

• keybase / python-triplesec
  A Python port of the triplesec library.
  ☆82Updated 2 years ago
• aaronsw / pytorctl
  ☆56Updated 2 years ago
• marklit / mass-ipv4-whois
  Mass IPv4 WHOIS Collection Tool
  ☆18Updated 9 years ago
• bbrodriges / pholcidae
  Tiny python web crawler
  ☆169Updated 9 years ago
• orf / simple
  Simple is a clone of Obtvse written in Python running on Flask.
  ☆510Updated 7 years ago
• getdnsapi / getdns-python-bindings
  Python bindings for getdns
  ☆26Updated 5 years ago
• collinalexbell / Email-Autoresponder
  ☆21Updated 10 years ago
• ssteuteville / scrapyz
  "Scrape Easy" - an extension of the Scrapy framework.
  ☆186Updated 9 years ago
• gtoonstra / remap
  MapReduce platform in python
  ☆34Updated 10 years ago
• arkaitzj / wsvnc
  In browser VNC client through websockets
  ☆32Updated 14 years ago
• andrewjw / celery-crawler
  A Django based search engine powered by CouchDB, celery and whoosh.
  ☆48Updated 10 years ago
• TeamHG-Memex / scrapy-dockerhub
  [UNMAINTAINED] Deploy, run and monitor your Scrapy spiders.
  ☆11Updated this week
• eklitzke / tornado-dns
  Asynchronous DNS lookups for Tornado
  ☆49Updated 10 years ago
• TeamHG-Memex / undercrawler
  A generic crawler
  ☆78Updated this week
• Alir3z4 / python-sanitize
  Bringing sanity to world of messed-up data
  ☆66Updated 11 years ago
• scrapinghub / exporters
  Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations
  ☆40Updated last year
• kakwa / pygraph_redis
  Simple python library to manipulate directed graphs in redis
  ☆23Updated 8 years ago
• Rafiot / Whois-Server
  Implementation of a Whois Server with a redis backend
  ☆15Updated 15 years ago
• twisted / vertex
  Vertex is a generalized, secure, peer-to-peer communications platform; formerly of the divmod.org project (https://code.launchpad.net/div…
  ☆75Updated 7 years ago
• staceysern / bt-twisted
  A BitTorrent client written in Python using Twisted
  ☆33Updated 11 years ago
• niklasb / webkit-server
  [not actively maintained] The C++ webkit-server from capybara-webkit with useful extensions and Python bindings
  ☆48Updated 5 years ago
• timmartin19 / ripozo-oasis
  A package and tutorial for turning your database into a HATEOAS ReSTful application
  ☆58Updated 10 years ago
• upgoingstar / TweetMonitor
  Keep an eye on specific keywords being posted on Twitter
  ☆46Updated 10 years ago
• atupal / ccrawler
  A distrubuted crawler ues celery.
  ☆15Updated 11 years ago
• Lispython / human_curl
  Simple Human wrapper for cURL library
  ☆201Updated 5 years ago
• mozilla / crontabber
  A cron job runner with self-healing and job dependencies.
  ☆134Updated 4 years ago
• shuque / pydig
  pydig: a DNS query tool written in Python
  ☆105Updated 8 months ago
• papaya-mobile / cyber
  A distributed asynchronous socket framework of Python
  ☆13Updated 9 years ago
• mitmproxy / netlib
  🗃️ This repository has been integrated into mitmproxy/mitmproxy.
  ☆66Updated 9 years ago
• gsko / DHTBot
  ☆44Updated 13 years ago