Alternatives and similar repositories for awesome-crawler:

Users that are interested in awesome-crawler are comparing it to the libraries listed below

• lorien / awesome-web-scraping
  List of libraries, tools and APIs for web scraping and data processing.
  ☆6,892Updated last month
• geekan / scrapy-examples
  Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.
  ☆3,200Updated last year
• scrapy-plugins / scrapy-splash
  Scrapy+Splash for JavaScript integration
  ☆3,182Updated last week
• DormyMo / SpiderKeeper
  admin ui for scrapy/open source scrapinghub
  ☆2,751Updated last year
• scrapinghub / portia
  Visual scraping for Scrapy
  ☆9,352Updated 7 months ago
• scrapinghub / splash
  Lightweight, scriptable browser as a service with an HTTP API
  ☆4,123Updated 6 months ago
• ChromeDevTools / awesome-chrome-devtools
  Awesome tooling and resources in the Chrome DevTools & DevTools Protocol ecosystem
  ☆6,249Updated last month
• scrapinghub / frontera
  A scalable frontier for web crawlers
  ☆1,307Updated 2 weeks ago
• scrapy / scrapyd
  A service daemon to run Scrapy spiders
  ☆3,002Updated 2 weeks ago
• dhamaniasad / HeadlessBrowsers
  A list of (almost) all headless web browsers in existence
  ☆6,311Updated 8 months ago
• binux / pyspider
  A Powerful Spider(Web Crawler) System in Python.
  ☆16,541Updated 9 months ago
• k4m4 / movies-for-hackers
  🎬 A curated list of movies every hacker & cyberpunk must watch.
  ☆10,746Updated 6 months ago
• marmelab / awesome-rest
  A collaborative list of great resources about RESTful API architecture, development, test, and performance
  ☆3,686Updated last week
• cjbarber / ToolsOfTheTrade
  Tools of The Trade, from Hacker News.
  ☆16,665Updated 6 months ago
• lorien / grab
  Web Scraping Framework
  ☆2,401Updated 11 months ago
• yujiosaka / headless-chrome-crawler
  Distributed crawler powered by Headless Chrome
  ☆5,549Updated last year
• elliotgao2 / toapi
  Every web site provides APIs.
  ☆3,516Updated 2 years ago
• transitive-bullshit / awesome-puppeteer
  A curated list of awesome puppeteer resources.
  ☆2,439Updated 7 months ago
• mezod / awesome-indie
  Resources for independent developers to make money
  ☆10,097Updated 8 months ago
• scrapy / scrapely
  A pure-python HTML screen-scraping library
  ☆1,870Updated 2 years ago
• grangier / python-goose
  Html Content / Article Extractor, web scrapping lib in Python
  ☆4,008Updated 3 years ago
• danburzo / percollate
  A command-line tool to turn web pages into readable PDF, EPUB, HTML, or Markdown docs.
  ☆4,370Updated last month
• rekihattori / awesome-programmers
  A list of history's greatest software engineers and tech pioneers
  ☆2,494Updated 3 years ago
• DopplerHQ / awesome-bots
  The most awesome list about bots ⭐️🤖
  ☆3,883Updated 7 months ago
• carpedm20 / awesome-hacking
  A curated list of awesome Hacking tutorials, tools and resources
  ☆13,651Updated 8 months ago
• marcobiedermann / search-engine-optimization
  🔍 A helpful checklist/collection of Search Engine Optimization (SEO) tips and techniques.
  ☆2,500Updated 9 months ago
• jmcarp / robobrowser
  ☆3,703Updated 4 years ago
• psf / requests-html
  Pythonic HTML Parsing for Humans™
  ☆13,780Updated 10 months ago
• hal9ai / awesome-dataviz
  A curated list of awesome data visualization libraries and resources.
  ☆3,880Updated last year
• bda-research / node-crawler
  Web Crawler/Spider for NodeJS + server-side jQuery ;-)
  ☆6,733Updated 2 months ago