Alternatives and similar repositories for scrapy-distributed

Users that are interested in scrapy-distributed are comparing it to the libraries listed below

• clemfromspace / scrapy-puppeteer
  Scrapy + Puppeteer
  ☆110Updated 4 years ago
• my8100 / logparser
  A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.
  ☆92Updated 11 months ago
• Gerapy / GerapyAutoExtractor
  Auto Extractor Module
  ☆332Updated last year
• Python3WebSpider / ScrapyRedisBloomFilter
  Scrapy Redis Bloom Filter
  ☆176Updated 4 years ago
• crawlab-team / webspot
  An intelligent web service to automatically detect web content and extract information from it.
  ☆85Updated 2 years ago
• RedisBloom / redisbloom-py
  Python client for Redisbloom
  ☆78Updated 2 years ago
• Gerapy / GerapyPlaywright
  Downloader Middleware to support Playwright in Scrapy & Gerapy
  ☆111Updated 3 years ago
• elacuesta / scrapy-pyppeteer
  Pyppeteer integration for Scrapy
  ☆58Updated 4 years ago
• Gerapy / GerapyPyppeteer
  Downloader Middleware to support Pyppeteer in Scrapy & Gerapy
  ☆136Updated 3 years ago
• ConlinH / aio-scrapy
  Implement scrapy with asyncio
  ☆71Updated 3 months ago
• scrapinghub / scrapy-frontera
  More flexible and featured Frontera scheduler for Scrapy
  ☆36Updated 6 months ago
• ClericPy / ichrome
  Chrome controller for Humans, based on Chrome Devtools Protocol(CDP) and python3.7+.
  ☆258Updated last month
• MeiK2333 / pyppeteer_stealth
  ☆261Updated 4 years ago
• abo123456789 / leek
  Distributed task redisqueue(最简单python分布式函数调度框架)
  ☆65Updated 3 weeks ago
• scrapinghub / spidermon
  Scrapy Extension for monitoring spiders execution.
  ☆549Updated 7 months ago
• crawlab-team / crawlab-core
  Backend core modules for Crawlab
  ☆51Updated last year
• rangertaha / scrapy-prometheus-exporter
  Scrapy stats exporter for prometheus
  ☆21Updated 3 months ago
• GeneralNewsExtractor / GneList
  A chrome extension to get XPath of list items in webpage easily.
  ☆36Updated 3 years ago
• cxapython / discogs_aio_spider
  基于httpx的一个大型项目 ，爬取黑胶唱片网站 Discogs
  ☆102Updated 4 months ago
• tieyongjie / scrapy-fingerprint
  ☆69Updated 2 years ago
• synodriver / awsome_scrapy_utils
  awsome scrapy utils
  ☆58Updated last year
• scrapy-plugins / scrapy-zyte-smartproxy
  Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
  ☆366Updated 8 months ago
• tenlee2012 / scrapy-kafka-redis
  Distributed crawling/scraping, Kafka And Redis based components for Scrapy
  ☆45Updated 5 years ago
• Python3WebSpider / AdslProxy
  Adsl Proxy Pool
  ☆238Updated 2 years ago
• aox-lei / scrapy-rabbitmq-scheduler
  rabbitmq的scrapy分布式爬虫
  ☆35Updated 4 years ago
• kingname / Tinepeas
  Tinepeas，我们自己的爬虫框架。
  ☆61Updated last year
• Granitosaurus / playwright-stealth
  ☆143Updated 2 years ago
• scrapy-plugins / scrapy-deltafetch
  Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
  ☆276Updated 9 months ago
• inlike / CookiePool
  一个强大的Cookie池项目，融合scrapy/requests/chrome储存cookie/cookie字符串/selenium等cookie形式
  ☆233Updated 5 years ago
• ispras / scrapy-puppeteer
  Library that helps use puppeteer in scrapy.
  ☆52Updated 4 months ago