Alternatives and similar repositories for awesome-crawler

Users that are interested in awesome-crawler are comparing it to the libraries listed below

• scrapinghub / portia
  Visual scraping for Scrapy
  ☆9,439Updated last year
• scrapinghub / splash
  Lightweight, scriptable browser as a service with an HTTP API
  ☆4,166Updated last year
• lorien / awesome-web-scraping
  List of libraries, tools and APIs for web scraping and data processing.
  ☆7,120Updated 7 months ago
• geekan / scrapy-examples
  Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.
  ☆3,248Updated last year
• scrapy-plugins / scrapy-splash
  Scrapy+Splash for JavaScript integration
  ☆3,218Updated 6 months ago
• DormyMo / SpiderKeeper
  admin ui for scrapy/open source scrapinghub
  ☆2,769Updated 2 years ago
• yujiosaka / headless-chrome-crawler
  Distributed crawler powered by Headless Chrome
  ☆5,588Updated 2 years ago
• ChromeDevTools / awesome-chrome-devtools
  Awesome tooling and resources in the Chrome DevTools & DevTools Protocol ecosystem
  ☆6,503Updated 7 months ago
• aivarsk / scrapy-proxies
  Random proxy middleware for Scrapy
  ☆1,670Updated 5 years ago
• scrapy / scrapyd
  A service daemon to run Scrapy spiders
  ☆3,056Updated this week
• dhamaniasad / HeadlessBrowsers
  A list of (almost) all headless web browsers in existence
  ☆6,414Updated 4 months ago
• lorien / grab
  Web Scraping Framework
  ☆2,407Updated last year
• transitive-bullshit / awesome-puppeteer
  A curated list of awesome puppeteer resources.
  ☆2,511Updated last year
• rekihattori / awesome-programmers
  A list of history's greatest software engineers and tech pioneers
  ☆2,521Updated 4 years ago
• elliotgao2 / toapi
  Every web site provides APIs.
  ☆3,530Updated 3 years ago
• marcobiedermann / search-engine-optimization
  🔍 A helpful checklist/collection of Search Engine Optimization (SEO) tips and techniques.
  ☆2,575Updated 5 months ago
• qinxuye / cola
  A high-level distributed crawling framework.
  ☆1,507Updated 3 years ago
• grangier / python-goose
  Html Content / Article Extractor, web scrapping lib in Python
  ☆4,046Updated 3 years ago
• elliotgao2 / gain
  Web crawling framework based on asyncio.
  ☆2,030Updated 6 years ago
• t3chnoboy / awesome-awesome-awesome
  A a curated list of curated lists of awesome lists.
  ☆2,044Updated last year
• mezod / awesome-indie
  Resources for independent developers to make money
  ☆10,433Updated last year
• neiesc / awesome-minimalist
  A curated list of awesome minimalist frameworks (simple and lightweight).
  ☆3,625Updated last month
• miyakogi / pyppeteer
  Headless chrome/chromium automation library (unofficial port of puppeteer)
  ☆3,571Updated 4 years ago
• my8100 / scrapydweb
  Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI.…
  ☆3,324Updated 5 months ago
• emijrp / awesome-awesome
  A curated list of awesome curated lists of many topics.
  ☆2,998Updated last year
• oxnr / awesome-analytics
  A curated list of analytics frameworks, software and other tools.
  ☆4,116Updated last year
• awesome-bootstrap-org / awesome-bootstrap
  ✨ Awesome - A curated list of amazing Bootstrap tools and themes.
  ☆1,400Updated last year
• DopplerHQ / awesome-bots
  The most awesome list about bots ⭐️🤖
  ☆3,976Updated last year
• apify / crawlee
  Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …
  ☆18,742Updated this week
• buriy / python-readability
  fast python port of arc90's readability tool, updated to match latest readability.js!
  ☆2,834Updated 3 months ago