fabienvauchelles / scrapoxy
Scrapoxy is a super proxy aggregator, allowing you to manage all proxies in one place 🎯, rather than spreading it across multiple scrapers 🕸️. It also smartly handles traffic routing 🔀 to minimize bans and increase success rates 🚀.
☆2,046Updated this week
Related projects ⓘ
Alternatives and complementary repositories for scrapoxy
- HTTP API for Scrapy spiders☆835Updated 4 months ago
- use multiple proxies with Scrapy☆739Updated 2 years ago
- Node.js implementation of a proxy server (think Squid) with support for SSL, authentication and upstream proxy chaining.☆850Updated this week
- This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.☆1,182Updated last year
- A pure-python HTML screen-scraping library☆1,863Updated 2 years ago
- Lightweight, scriptable browser as a service with an HTTP API☆4,099Updated 3 months ago
- A service daemon to run Scrapy spiders☆2,970Updated last week
- A curated list of awesome packages, articles, and other cool resources from the Scrapy community.☆537Updated last year
- The web scraper that's nearly impossible to block - now called @ulixee/hero☆674Updated last year
- Scrapy+Splash for JavaScript integration☆3,157Updated last year
- Random proxy middleware for Scrapy☆1,658Updated 5 years ago
- A scalable frontier for web crawlers☆1,302Updated last year
- Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls☆267Updated 3 years ago
- Distributed crawling infrastructure running on top of severless computation, cloud storage (such as S3) and sophisticated queues.☆415Updated last year
- Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprint…☆4,145Updated 4 months ago
- Browser fingerprinting tools for anonymizing your scrapers. Developed by Apify.☆987Updated this week
- Rotating TOR proxy with Docker☆1,139Updated 6 months ago
- Visual scraping for Scrapy☆9,303Updated 4 months ago
- ☆539Updated 8 months ago
- Scrapy Extension for monitoring spiders execution.☆533Updated last week
- Command line client for Scrapyd server☆770Updated last month
- The web browser built for scraping☆951Updated this week
- Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS☆3,865Updated 8 months ago
- admin ui for scrapy/open source scrapinghub☆2,741Updated last year
- Luminati HTTP/HTTPS Proxy manager☆744Updated 2 months ago
- Is headless chrome currently detectable? Let's pit the detections and detection evasions against eachother.☆646Updated 3 years ago
- Random User-Agent middleware based on fake-useragent☆687Updated last year
- Extract embedded metadata from HTML markup☆853Updated last week
- Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy☆356Updated last week
- 🎭 Playwright integration for Scrapy☆1,030Updated last week