Alternatives and similar repositories for awesome-crawler

Users that are interested in awesome-crawler are comparing it to the libraries listed below

• lorien / awesome-web-scraping
  List of libraries, tools and APIs for web scraping and data processing.
  ☆7,077Updated 6 months ago
• dhamaniasad / HeadlessBrowsers
  A list of (almost) all headless web browsers in existence
  ☆6,401Updated 3 months ago
• scrapinghub / portia
  Visual scraping for Scrapy
  ☆9,432Updated last year
• yujiosaka / headless-chrome-crawler
  Distributed crawler powered by Headless Chrome
  ☆5,581Updated 2 years ago
• scrapinghub / splash
  Lightweight, scriptable browser as a service with an HTTP API
  ☆4,164Updated 11 months ago
• DormyMo / SpiderKeeper
  admin ui for scrapy/open source scrapinghub
  ☆2,767Updated 2 years ago
• elliotgao2 / toapi
  Every web site provides APIs.
  ☆3,530Updated 3 years ago
• rekihattori / awesome-programmers
  A list of history's greatest software engineers and tech pioneers
  ☆2,521Updated 4 years ago
• ChromeDevTools / awesome-chrome-devtools
  Awesome tooling and resources in the Chrome DevTools & DevTools Protocol ecosystem
  ☆6,475Updated 6 months ago
• marcobiedermann / search-engine-optimization
  🔍 A helpful checklist/collection of Search Engine Optimization (SEO) tips and techniques.
  ☆2,570Updated 4 months ago
• scrapoxy / scrapoxy
  Scrapoxy is a super proxies manager that orchestrates all your proxies into one place, rather than spreading management across multiple s…
  ☆2,314Updated last month
• ramnes / awesome-mongodb
  A curated list of awesome MongoDB resources, libraries, tools and applications
  ☆2,567Updated last month
• geekan / scrapy-examples
  Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.
  ☆3,245Updated last year
• marmelab / awesome-rest
  A collaborative list of great resources about RESTful API architecture, development, test, and performance
  ☆3,767Updated 5 months ago
• mezod / awesome-indie
  Resources for independent developers to make money
  ☆10,392Updated last year
• oxnr / awesome-bigdata
  A curated list of awesome big data frameworks, ressources and other awesomeness.
  ☆13,726Updated 5 months ago
• lorien / grab
  Web Scraping Framework
  ☆2,405Updated last year
• mfornos / awesome-microservices
  A curated list of Microservice Architecture related principles and technologies.
  ☆13,758Updated 6 months ago
• scrapy-plugins / scrapy-splash
  Scrapy+Splash for JavaScript integration
  ☆3,218Updated 5 months ago
• emijrp / awesome-awesome
  A curated list of awesome curated lists of many topics.
  ☆2,986Updated 11 months ago
• grangier / python-goose
  Html Content / Article Extractor, web scrapping lib in Python
  ☆4,042Updated 3 years ago
• scrapinghub / frontera
  A scalable frontier for web crawlers
  ☆1,313Updated last month
• transitive-bullshit / awesome-puppeteer
  A curated list of awesome puppeteer resources.
  ☆2,502Updated last year
• scrapy / scrapely
  A pure-python HTML screen-scraping library
  ☆1,879Updated 3 years ago
• binux / pyspider
  A Powerful Spider(Web Crawler) System in Python.
  ☆16,715Updated last year
• aivarsk / scrapy-proxies
  Random proxy middleware for Scrapy
  ☆1,670Updated 5 years ago
• thibmaek / awesome-raspberry-pi
  📝 A curated list of awesome Raspberry Pi tools, projects, images and resources
  ☆14,778Updated last month
• istresearch / scrapy-cluster
  This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
  ☆1,210Updated last year
• DopplerHQ / awesome-bots
  The most awesome list about bots ⭐️🤖
  ☆3,972Updated last year
• Gerapy / Gerapy
  Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
  ☆3,466Updated 8 months ago