clemfromspace / scrapy-selenium
Scrapy middleware to handle javascript pages using selenium
β922Updated 4 months ago
Related projects β
Alternatives and complementary repositories for scrapy-selenium
- Command line client for Scrapyd serverβ770Updated last month
- π Playwright integration for Scrapyβ1,030Updated last week
- Random User-Agent middleware based on fake-useragentβ687Updated last year
- Scrapy+Splash for JavaScript integrationβ3,157Updated last year
- Random proxy middleware for Scrapyβ1,658Updated 5 years ago
- use multiple proxies with Scrapyβ739Updated 2 years ago
- Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapyβ356Updated last week
- β164Updated 4 years ago
- A service daemon to run Scrapy spidersβ2,970Updated last week
- HTTP API for Scrapy spidersβ835Updated 4 months ago
- Splash + HAProxy + Docker Composeβ198Updated 5 years ago
- Scrapy spider middleware to ignore requests to pages containing items seen in previous crawlsβ267Updated 3 years ago
- This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.β1,182Updated last year
- Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI.β¦β3,161Updated last month
- A curated list of awesome packages, articles, and other cool resources from the Scrapy community.β537Updated last year
- Lightweight, scriptable browser as a service with an HTTP APIβ4,099Updated 3 months ago
- A scalable frontier for web crawlersβ1,302Updated last year
- A Python wrapper for working with Scrapyd's API.β268Updated 3 months ago
- Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.jsβ3,357Updated 3 weeks ago
- Scrapy Book Codeβ480Updated 6 years ago
- Scrapy Extension for monitoring spiders execution.β533Updated last week
- Parsel lets you extract data from XML/HTML documents using XPath or CSS selectorsβ1,148Updated last month
- A Scrapy middleware to bypass the CloudFlare's anti-bot protectionβ106Updated 3 years ago
- Parsing JavaScript objects into Python data structuresβ197Updated last month
- Scrapy extension to control spiders using JSON-RPCβ296Updated 5 years ago
- Extends Selenium WebDriver classes to include the request function from the Requests library, while doing all the needed cookie and requeβ¦β494Updated 8 months ago
- playwright stealthβ540Updated 3 months ago
- Trying to make python selenium more stealthy.β648Updated 2 years ago
- Extends Selenium's Python bindings to give you the ability to inspect requests made by the browser.β1,915Updated 10 months ago
- admin ui for scrapy/open source scrapinghubβ2,741Updated last year