wind2sing / aCrawler
π A powerful web-crawling framework, based on aiohttp.
β15Updated 5 years ago
Alternatives and similar repositories for aCrawler:
Users that are interested in aCrawler are comparing it to the libraries listed below
- Pyppeteer integration for Scrapyβ58Updated 4 years ago
- Async wrapper for requests / aiohttp, and some crawler toolkits. Let synchronization code enjoy the performance of asynchronous programmiβ¦β24Updated 3 months ago
- Use pyppeteer from a Scrapy spiderβ59Updated 5 years ago
- python 3.7 asyncio tutorial.β14Updated 5 years ago
- Asyncio web crawling framework. Work in progress.β18Updated 8 months ago
- Analyze scraped dataβ46Updated 5 years ago
- Live video streaming server built with Sanic framework and opencv-python.β12Updated 7 years ago
- Library for annotation-based dependency injectionβ21Updated 3 months ago
- Web scraping Page Objects core libraryβ99Updated 2 months ago
- Page Object pattern for Scrapyβ121Updated 2 months ago
- A session-management extension for Scrapy.β10Updated last year
- simple motor wrapper for sanicβ57Updated 2 years ago
- β29Updated 4 years ago
- tus.io protocol implementation for aiohttp.web applicationsβ17Updated 2 years ago
- Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.β56Updated 3 years ago
- A complimentary proxy to help to use SPM with headless browsersβ108Updated last year
- A Ruia plugin for loading javascript - pyppeteerβ18Updated 3 years ago
- Simple Web UI for Scrapy spider management via Scrapydβ51Updated 6 years ago
- Pre-built Scrapy spiders for AutoExtractβ19Updated last year
- Scrapy + Puppeteerβ110Updated 3 years ago
- Lightweight library that converts a HTML webpage to JSON data using a template defined in JSON.β22Updated last week
- Library that helps use puppeteer in scrapy.β52Updated 3 weeks ago
- Free & open source API service for obtaining information about +9600 universities worldwide.β67Updated 3 years ago
- Library to populate items using XPath and CSS with a convenient APIβ48Updated last month
- Scrapy schema validation pipeline and Item builder using JSON Schemaβ44Updated 4 years ago
- sonic search backend client in pythonβ60Updated 4 years ago
- A simple python tool that generates a requests/bs4 based web scraperβ26Updated 2 years ago
- Scrapy middleware which allows to crawl only new contentβ80Updated 2 years ago
- More flexible and featured Frontera scheduler for Scrapyβ37Updated 5 months ago
- A simple, Qt-Webengine powered web browser with built in functionality for basic scrapy webscraping support.β109Updated 11 months ago