Alternatives and similar repositories for crawler

Users that are interested in crawler are comparing it to the libraries listed below

• bbrodriges / pholcidae
  Tiny python web crawler
  ☆169Updated 9 years ago
• aaronsw / pytorctl
  ☆56Updated 2 years ago
• ssteuteville / scrapyz
  "Scrape Easy" - an extension of the Scrapy framework.
  ☆186Updated 9 years ago
• getdnsapi / getdns-python-bindings
  Python bindings for getdns
  ☆26Updated 5 years ago
• jessamynsmith / twitterbot
  Configurable bot that replies to mentions and posts messages to twitter
  ☆19Updated 4 years ago
• TeamHG-Memex / scrapy-dockerhub
  [UNMAINTAINED] Deploy, run and monitor your Scrapy spiders.
  ☆11Updated 10 years ago
• Alir3z4 / python-sanitize
  Bringing sanity to world of messed-up data
  ☆66Updated 11 years ago
• keybase / python-triplesec
  A Python port of the triplesec library.
  ☆82Updated last year
• scrapinghub / exporters
  Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations
  ☆40Updated last year
• andrewjw / celery-crawler
  A Django based search engine powered by CouchDB, celery and whoosh.
  ☆49Updated 10 years ago
• TeamHG-Memex / undercrawler
  A generic crawler
  ☆78Updated 7 years ago
• teddziuba / pqauth
  Web API Authentication with SSH Public Keys
  ☆161Updated 12 years ago
• walkr / nanoservice
  nanoservice is a small Python library for writing lightweight networked services using nanomsg
  ☆31Updated 10 years ago
• orf / simple
  Simple is a clone of Obtvse written in Python running on Flask.
  ☆510Updated 7 years ago
• atl / thrifty-p2p
  A very simple python peer-to-peer implementation using consistent hashing (and an example distributed key-value store layered atop it) us…
  ☆30Updated 16 years ago
• papaya-mobile / cyber
  A distributed asynchronous socket framework of Python
  ☆13Updated 9 years ago
• atupal / ccrawler
  A distrubuted crawler ues celery.
  ☆16Updated 11 years ago
• TeamHG-Memex / autologin
  A project to attempt to automatically login to a website given a single seed
  ☆127Updated 2 months ago
• collinalexbell / Email-Autoresponder
  ☆21Updated 10 years ago
• seomoz / g-crawl-py
  Gevent Crawling in Python, with Utilities
  ☆22Updated 10 years ago
• kakwa / pygraph_redis
  Simple python library to manipulate directed graphs in redis
  ☆23Updated 8 years ago
• gtoonstra / remap
  MapReduce platform in python
  ☆34Updated 10 years ago
• scrapinghub / webpager
  Paginating the web
  ☆37Updated 11 years ago
• TeamHG-Memex / autologin-middleware
  Scrapy middleware for the autologin
  ☆36Updated 7 years ago
• balanced / balanced-python
  Balanced API library in python.
  ☆70Updated 4 years ago
• tryolabs / daywatch
  E-commerce scraping and analytics platform.
  ☆52Updated 10 years ago
• asdofindia / python-telegram-bot
  This is a telegram bot written in python. It uses the CLI of telegram by vysheng to connect. No longer developed. Checkout
  ☆30Updated 11 years ago
• discoproject / discodb
  An efficient, immutable, persistent mapping object
  ☆98Updated 6 years ago
• scrapinghub / scrapylib
  Collection of Scrapy utilities (extensions, middlewares, pipelines, etc)
  ☆33Updated 7 years ago
• staceysern / bt-twisted
  A BitTorrent client written in Python using Twisted
  ☆33Updated 11 years ago