Alternatives and similar repositories for xspider:

Users that are interested in xspider are comparing it to the libraries listed below

• scrapinghub / autologin
  A project to attempt to automatically login to a website given a single seed
  ☆11Updated 8 months ago
• xelzmm / proxy_server_crawler
  an awesome public proxy server crawler based on scrapy framework
  ☆96Updated 7 years ago
• TeamHG-Memex / arachnado
  Web Crawling UI and HTTP API, based on Scrapy and Tornado
  ☆162Updated 2 years ago
• julien-duponchelle / scrapy-graphite
  Output scrapy statistics to graphite/carbon
  ☆54Updated 11 years ago
• jmg / crawley
  Pythonic Crawling / Scraping Framework based on Non Blocking I/O operations.
  ☆189Updated last year
• scrapy-plugins / scrapy-deltafetch
  Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
  ☆269Updated 3 years ago
• KDF5000 / RSpider
  一个基于scrapy-redis的分布式爬虫模板
  ☆42Updated 7 years ago
• bufubaoni / antitools
  antitools
  ☆17Updated 4 years ago
• douban / brownant
  Brownant is a web data extracting framework.
  ☆159Updated 7 years ago
• hustcc / wrapcache
  A python Function / Method OUTPUT cache system base on function Decorators.
  ☆58Updated 4 years ago
• keepljg / flySpider
  基于 asyncio,aiohttp,uvloop 的爬虫框架
  ☆14Updated 6 years ago
• gmr / tornado-elasticsearch
  Extends the official Elasticsearch Python API adding Tornado AsyncHTTPClient support
  ☆38Updated 4 years ago
• djm / python-scrapyd-api
  A Python wrapper for working with Scrapyd's API.
  ☆270Updated 6 months ago
• TeamHG-Memex / scrapy-kafka-export
  Scrapy extension which writes crawled items to Kafka
  ☆30Updated 6 years ago
• lopuhin / scrapy-pyppeteer
  Use pyppeteer from a Scrapy spider
  ☆60Updated 5 years ago
• flisky / scrapy-phantomjs-downloader
  PhantomJS Downloader for Scrapy, Yeah!
  ☆94Updated 10 years ago
• feiskyer / scrapy-examples
  Some scrapy and web.py exmaples
  ☆79Updated 7 years ago
• runtangr / taoyoulue_spider
  基于mongodb存储，redis缓存，celery 实现的分布式爬虫。
  ☆13Updated 2 years ago
• matiasb / demiurge
  PyQuery-based scraping micro-framework.
  ☆116Updated 3 years ago
• liuliqiang / redisbeat
  celery redis scheduler, dynamic add/modify/delete task from celery.
  ☆180Updated 4 months ago
• xupeng / bloomfilter-redis
  Python bloom filter using redis as a shared backend.
  ☆19Updated 7 years ago
• seomoz / pyreBloom
  Fast Redis Bloom Filters in Python
  ☆289Updated 6 years ago
• scrapy-plugins / scrapy-headless
  ☆29Updated 3 years ago
• scrapinghub / testspiders
  Useful test spiders for Scrapy
  ☆185Updated 5 years ago
• SylvanasSun / FishFishJump
  Fish Fish Jump is a solution in the python that simply and basic for search engines.
  ☆55Updated 6 years ago
• roycehaynes / scrapy-rabbitmq
  A RabbitMQ Scheduler for Scrapy
  ☆86Updated 2 years ago
• ClericPy / torequests
  Async wrapper for requests / aiohttp, and some crawler toolkits. Let synchronization code enjoy the performance of asynchronous programmi…
  ☆24Updated 2 weeks ago
• timogasda / python-sqlparser
  A Python Module for the "General SQL Parser" library (sqlparser.com)
  ☆76Updated last year
• stefanw / scrapa
  Python 3 AsyncIO powered scraping framework with batteries included
  ☆20Updated 8 years ago
• aaldaber / Distributed-Multi-User-Scrapy-System-with-a-Web-UI
  Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner
  ☆113Updated 6 years ago