wind2sing / aCrawlerLinks
π A powerful web-crawling framework, based on aiohttp.
β15Updated 6 years ago
Alternatives and similar repositories for aCrawler
Users that are interested in aCrawler are comparing it to the libraries listed below
Sorting:
- Pyppeteer integration for Scrapyβ58Updated 4 years ago
- Async wrapper for requests / aiohttp, and some crawler toolkits. Let synchronization code enjoy the performance of asynchronous programmiβ¦β24Updated 10 months ago
- tus.io protocol implementation for aiohttp.web applicationsβ17Updated 3 years ago
- A complimentary proxy to help to use SPM with headless browsersβ108Updated 2 years ago
- Lightweight library that converts a HTML webpage to JSON data using a template defined in JSON.β23Updated 6 months ago
- python 3.7 asyncio tutorial.β14Updated 6 years ago
- Use pyppeteer from a Scrapy spiderβ59Updated 5 years ago
- Extract text from HTMLβ135Updated 5 years ago
- A session-management extension for Scrapy.β10Updated last year
- Scrapy + Puppeteerβ110Updated 4 years ago
- Asyncio web crawling framework. Work in progress.β19Updated last year
- A simple, Qt-Webengine powered web browser with built in functionality for basic scrapy webscraping support.β110Updated last year
- Web scraping Page Objects core libraryβ103Updated this week
- Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.β55Updated 3 years ago
- Analyze scraped dataβ46Updated 6 years ago
- Extract structured data from HTML and XML documents like a boss.β50Updated last year
- β28Updated 4 years ago
- Convert Javascript code to an XML documentβ187Updated 3 years ago
- A simple python tool that generates a requests/bs4 based web scraperβ27Updated 3 years ago
- A Requests-compatible interface for PycURL.β71Updated 2 months ago
- π π BitTorrent client built with Python + asyncioβ158Updated 2 years ago
- A scrapy extension to sync `.scrapy` folder to an S3 bucketβ17Updated 3 years ago
- Scraping tweets quickly using celery, RabbitMQ and Docker clusterβ50Updated 3 years ago
- A thin wrapper for aiohttp client with Requests simplicityβ55Updated last year
- Faster CSV for Pythonβ104Updated 3 years ago
- Library that helps use puppeteer in scrapy.β52Updated 4 months ago
- Python UI/Web Frameworkβ19Updated 2 years ago
- Combine XPath, CSS Selectors and JSONPath for Web data extracting.β29Updated 11 months ago
- A Ruia plugin for loading javascript - pyppeteerβ18Updated 3 years ago
- Splash + HAProxy + Docker Composeβ197Updated 7 years ago