scrapy-plugins / scrapy-playwright

Related projects ⓘ

Alternatives and complementary repositories for scrapy-playwright

• clemfromspace / scrapy-selenium
  Scrapy middleware to handle javascript pages using selenium
  ☆922Updated 4 months ago
• AtuboDad / playwright_stealth
  playwright stealth
  ☆540Updated 3 months ago
• scrapy / scrapyd-client
  Command line client for Scrapyd server
  ☆770Updated last month
• TeamHG-Memex / scrapy-rotating-proxies
  use multiple proxies with Scrapy
  ☆739Updated 2 years ago
• Nykakin / chompjs
  Parsing JavaScript objects into Python data structures
  ☆197Updated last month
• scrapinghub / spidermon
  Scrapy Extension for monitoring spiders execution.
  ☆533Updated last week
• scrapinghub / scrapyrt
  HTTP API for Scrapy spiders
  ☆835Updated 4 months ago
• scrapy-plugins / scrapy-splash
  Scrapy+Splash for JavaScript integration
  ☆3,157Updated last year
• rejoiceinhope / scrapy-proxy-pool
  ☆164Updated 4 years ago
• scrapy / parsel
  Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors
  ☆1,148Updated last month
• alecxe / scrapy-fake-useragent
  Random User-Agent middleware based on fake-useragent
  ☆687Updated last year
• scrapy-plugins / scrapy-zyte-smartproxy
  Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
  ☆356Updated last week
• rushter / selectolax
  Python binding to Modest and Lexbor engines (fast HTML5 parser with CSS selectors).
  ☆1,161Updated last week
• MeiK2333 / pyppeteer_stealth
  ☆252Updated 3 years ago
• istresearch / scrapy-cluster
  This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
  ☆1,182Updated last year
• scrapy / scrapyd
  A service daemon to run Scrapy spiders
  ☆2,970Updated last week
• clemfromspace / scrapy-cloudflare-middleware
  A Scrapy middleware to bypass the CloudFlare's anti-bot protection
  ☆106Updated 3 years ago
• jxlil / scrapy-impersonate
  Scrapy download handler that can impersonate browser' TLS signatures or JA3 fingerprints.
  ☆109Updated last month
• diprajpatra / selenium-stealth
  Trying to make python selenium more stealthy.
  ☆648Updated 2 years ago
• aivarsk / scrapy-proxies
  Random proxy middleware for Scrapy
  ☆1,658Updated 5 years ago
• wkeeling / selenium-wire
  Extends Selenium's Python bindings to give you the ability to inspect requests made by the browser.
  ☆1,915Updated 10 months ago
• Granitosaurus / playwright-stealth
  ☆124Updated last year
• Gerapy / Gerapy
  Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
  ☆3,357Updated 3 weeks ago
• AccordBox / awesome-scrapy
  A curated list of awesome packages, articles, and other cool resources from the Scrapy community.
  ☆537Updated last year
• daijro / browserforge
  🎭 Intelligent browser header & fingerprint generator
  ☆252Updated 5 months ago
• scrapinghub / splash
  Lightweight, scriptable browser as a service with an HTTP API
  ☆4,099Updated 3 months ago
• EasyPi / docker-scrapyd
  🕷️ Scrapyd is an application for deploying and running Scrapy spiders.
  ☆79Updated last week
• scrapinghub / scrapy-poet
  Page Object pattern for Scrapy
  ☆119Updated this week
• elacuesta / scrapy-pyppeteer
  Pyppeteer integration for Scrapy
  ☆60Updated 3 years ago
• pyppeteer / pyppeteer
  Headless chrome/chromium automation library (unofficial port of puppeteer)
  ☆3,690Updated 4 months ago