wind2sing / aCrawlerLinks
🔍 A powerful web-crawling framework, based on aiohttp.
☆15Updated 5 years ago
Alternatives and similar repositories for aCrawler
Users that are interested in aCrawler are comparing it to the libraries listed below
Sorting:
- Pyppeteer integration for Scrapy☆58Updated 4 years ago
- Async wrapper for requests / aiohttp, and some crawler toolkits. Let synchronization code enjoy the performance of asynchronous programmi…☆24Updated 5 months ago
- Scrapy + Puppeteer☆110Updated 4 years ago
- python 3.7 asyncio tutorial.☆14Updated 5 years ago
- tus.io protocol implementation for aiohttp.web applications☆17Updated 2 years ago
- Zyte Automatic Extraction integration for Scrapy☆56Updated 3 years ago
- A complimentary proxy to help to use SPM with headless browsers☆108Updated 2 years ago
- Use pyppeteer from a Scrapy spider☆59Updated 5 years ago
- More flexible and featured Frontera scheduler for Scrapy☆37Updated last month
- Asyncio web crawling framework. Work in progress.☆19Updated 11 months ago
- Extract structured data from HTML and XML documents like a boss.☆49Updated 7 months ago
- Page Object pattern for Scrapy☆123Updated last week
- Web scraping Page Objects core library☆102Updated 2 weeks ago
- A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.☆93Updated 6 months ago
- A scrapy extension to sync `.scrapy` folder to an S3 bucket☆17Updated 3 years ago
- Simple Web UI for Scrapy spider management via Scrapyd☆51Updated 7 years ago
- Library to populate items using XPath and CSS with a convenient API☆48Updated 2 weeks ago
- A RabbitMQ Scheduler for Scrapy☆87Updated 2 years ago
- A simple, Qt-Webengine powered web browser with built in functionality for basic scrapy webscraping support.☆109Updated last year
- Lightweight library that converts a HTML webpage to JSON data using a template defined in JSON.☆23Updated last month
- A Scrapy middleware to bypass the CloudFlare's anti-bot protection☆110Updated 4 years ago
- Trio driver for Chrome DevTools Protocol (CDP)☆68Updated 3 years ago
- A decorator to write coroutine-like spider callbacks.☆109Updated 2 years ago
- Fast Indexed python HTML parser which builds a DOM node tree, providing common getElementsBy* functions for scraping, testing, modificati…☆102Updated 2 years ago
- Splash + HAProxy + Docker Compose☆197Updated 6 years ago
- Pre-built Scrapy spiders for AutoExtract☆19Updated last year
- Scrapy integration with Tor for anonymous web scraping☆46Updated 9 years ago
- CoCrawler is a versatile web crawler built using modern tools and concurrency.☆191Updated 3 years ago
- A Ruia plugin for loading javascript - pyppeteer☆18Updated 3 years ago
- An efficient and lightweight thread pool☆38Updated 4 years ago