Alternatives and similar repositories for crawley

Users that are interested in crawley are comparing it to the libraries listed below

• scrapinghub / scrapylib
  Collection of Scrapy utilities (extensions, middlewares, pipelines, etc)
  ☆33Updated 7 years ago
• TeamHG-Memex / arachnado
  Web Crawling UI and HTTP API, based on Scrapy and Tornado
  ☆160Updated 2 weeks ago
• scrapy-plugins / scrapy-jsonrpc
  Scrapy extension to control spiders using JSON-RPC
  ☆300Updated 6 years ago
• scrapy / w3lib
  Python library of web-related functions
  ☆413Updated last month
• brandicted / scrapy-webdriver
  ☆143Updated 10 years ago
• scrapinghub / testspiders
  Useful test spiders for Scrapy
  ☆184Updated 6 years ago
• ask / flask-celery
  Celery integration for Flask (SINCE CELERY 3.0 THIS IS NO LONGER NEEDED)
  ☆200Updated 12 years ago
• deldotdr / txRedis
  A Redis client library for Twisted Python
  ☆127Updated 4 years ago
• matiasb / demiurge
  PyQuery-based scraping micro-framework.
  ☆118Updated 4 years ago
• retresco / Spyder
  A Python web crawler using Tornado and ZeroMQ
  ☆139Updated 13 years ago
• julien-duponchelle / scrapy-elasticsearch
  A scrapy pipeline which send items to Elastic Search server
  ☆98Updated 8 years ago
• sebdah / scrapy-mongodb
  MongoDB pipeline for Scrapy. This module supports both MongoDB in standalone setups and replica sets. scrapy-mongodb will insert the item…
  ☆358Updated 4 years ago
• iamteem / redisco
  A Python Library for Simple Models and Containers Persisted in Redis
  ☆300Updated 9 years ago
• andreisavu / mongodb-log
  MongoDB python logging handler. Python centralized logging made easy.
  ☆197Updated 13 years ago
• davisp / python-spidermonkey
  Python/JavaScript bridge module, making use of Mozilla's spidermonkey JavaScript implementation.
  ☆304Updated 8 years ago
• twisted / txmongo
  asynchronous python driver for mongo
  ☆340Updated 3 months ago
• cnu / scrapy-random-useragent
  Scrapy Middleware to set a random User-Agent for every Request.
  ☆202Updated 6 years ago
• pyelasticsearch / pyelasticsearch
  python elasticsearch client
  ☆362Updated 3 years ago
• mitsuhiko / bf3-aggregator
  Sourcecode for the bf3 developer news aggregator.
  ☆82Updated 14 years ago
• andrewjw / celery-crawler
  A Django based search engine powered by CouchDB, celery and whoosh.
  ☆48Updated 10 years ago
• aparo / pyes
  Python connector for ElasticSearch - the pythonic way to use ElasticSearch
  ☆606Updated 4 years ago
• benoitc / restkit
  an HTTP resource kit for Python
  ☆403Updated 4 years ago
• truemped / tornadotools
  A collection of tools for Tornado
  ☆63Updated 11 years ago
• douban / brownant
  Brownant is a web data extracting framework.
  ☆158Updated 8 years ago
• mdipierro / workflow
  minimalist file based workflow
  ☆565Updated 9 years ago
• niwinz / phantompy
  Phantompy is a headless WebKit engine with powerful pythonic api build on top of Qt5 Webkit
  ☆612Updated 8 years ago
• djangsters / frequests
  Requests + futures = <3 - a grequests fork, origonal code: https://github.com/kennethreitz/grequests
  ☆63Updated 10 years ago
• flisky / scrapy-phantomjs-downloader
  PhantomJS Downloader for Scrapy, Yeah!
  ☆94Updated 11 years ago
• timbertson / python-readability
  [abandoned] python port of arc90's readability bookmarklet
  ☆542Updated 14 years ago
• marcelnicolay / torneira
  Torneira is a lightweight and rapid web framework build on top of Tornado
  ☆69Updated 12 years ago