wind2sing / aCrawlerLinks
🔍 A powerful web-crawling framework, based on aiohttp.
☆15Updated 6 years ago
Alternatives and similar repositories for aCrawler
Users that are interested in aCrawler are comparing it to the libraries listed below
Sorting:
- Pyppeteer integration for Scrapy☆58Updated 4 years ago
- Async wrapper for requests / aiohttp, and some crawler toolkits. Let synchronization code enjoy the performance of asynchronous programmi…☆24Updated 11 months ago
- Use pyppeteer from a Scrapy spider☆59Updated 6 years ago
- Lightweight library that converts a HTML webpage to JSON data using a template defined in JSON.☆23Updated 8 months ago
- Analyze scraped data☆46Updated 6 years ago
- python 3.7 asyncio tutorial.☆14Updated 6 years ago
- Scrapy + Puppeteer☆110Updated 4 years ago
- Asyncio web crawling framework. Work in progress.☆19Updated last year
- Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.☆57Updated 3 years ago
- Zyte Automatic Extraction integration for Scrapy☆56Updated 4 years ago
- Extract structured data from HTML and XML documents like a boss.☆50Updated last year
- Scrapy schema validation pipeline and Item builder using JSON Schema☆45Updated 4 years ago
- tus.io protocol implementation for aiohttp.web applications☆17Updated 3 years ago
- Simple Web UI for Scrapy spider management via Scrapyd☆50Updated 7 years ago
- CoCrawler is a versatile web crawler built using modern tools and concurrency.☆193Updated 3 years ago
- Library for annotation-based dependency injection☆24Updated last month
- A complimentary proxy to help to use SPM with headless browsers☆108Updated 2 years ago
- A Ruia plugin for loading javascript - pyppeteer☆18Updated 3 years ago
- Web scraping Page Objects core library☆104Updated last week
- A simple, Qt-Webengine powered web browser with built in functionality for basic scrapy webscraping support.☆110Updated last year
- Extract text from HTML☆134Updated 2 weeks ago
- Page Object pattern for Scrapy☆126Updated last week
- Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner☆112Updated 7 years ago
- Combine XPath, CSS Selectors and JSONPath for Web data extracting.☆29Updated last year
- A Flask-based CMS / web application for compatibility and simplicity.☆14Updated 6 years ago
- Scrapy middleware for the autologin☆36Updated 2 weeks ago
- scrapy mysql pipeline☆49Updated 4 years ago
- A scrapy extension to sync `.scrapy` folder to an S3 bucket☆18Updated 3 years ago
- Python clients for Zyte AutoExtract API☆41Updated 4 years ago
- A RabbitMQ Scheduler for Scrapy☆87Updated 3 years ago