wind2sing / aCrawler
π A powerful web-crawling framework, based on aiohttp.
β15Updated 5 years ago
Alternatives and similar repositories for aCrawler:
Users that are interested in aCrawler are comparing it to the libraries listed below
- Async wrapper for requests / aiohttp, and some crawler toolkits. Let synchronization code enjoy the performance of asynchronous programmiβ¦β24Updated last year
- Lightweight library that converts a HTML webpage to JSON data using a template defined in JSON.β21Updated 4 years ago
- python 3.7 asyncio tutorial.β14Updated 5 years ago
- Pyppeteer integration for Scrapyβ59Updated 3 years ago
- Combine XPath, CSS Selectors and JSONPath for Web data extracting.β27Updated 3 weeks ago
- Use pyppeteer from a Scrapy spiderβ60Updated 4 years ago
- Asyncio web crawling framework. Work in progress.β18Updated 5 months ago
- tus.io protocol implementation for aiohttp.web applicationsβ16Updated 2 years ago
- sonic search backend client in pythonβ60Updated 3 years ago
- Mobilenium allows you to use Selenium and have access to status codes and HTTP headers, without the need for manual labor.β21Updated 5 years ago
- β29Updated 3 years ago
- scrapy mysql pipelineβ49Updated 3 years ago
- Pre-built Scrapy spiders for AutoExtractβ19Updated 8 months ago
- Easily document your Sanic API with Swagger UI, Plus param validation and model serialization.β47Updated 3 years ago
- Asyncio interface for Peewee ORMβ46Updated 4 years ago
- Zyte Automatic Extraction integration for Scrapyβ56Updated 2 years ago
- A Ruia plugin for loading javascript - pyppeteerβ18Updated 2 years ago
- A Requests-compatible interface for PycURL.β64Updated 5 months ago
- Scrapy + Puppeteerβ111Updated 3 years ago
- Crochet-based blocking API for Scrapy.β46Updated 7 years ago
- Price and currency parsing utilityβ26Updated last year
- Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.β56Updated 2 years ago
- Extract structured data from HTML and XML documents like a boss.β50Updated last month
- A query expression for extracting data from JSON.β40Updated 3 weeks ago
- Analyze scraped dataβ47Updated 5 years ago
- A complimentary proxy to help to use SPM with headless browsersβ109Updated last year
- AintQ Is Not Task Queue - a Python asyncio task queue on PostgreSQL.β50Updated 2 years ago
- Simple Flask scheduled tasks without extra daemonsβ101Updated 3 years ago
- Simple Web UI for Scrapy spider management via Scrapydβ51Updated 6 years ago