hugsbrugs / scrapyd-webapp
Scrapyd web application for managing projects, spiders and visualize jobs logs and items
☆14Updated 9 years ago
Alternatives and similar repositories for scrapyd-webapp:
Users that are interested in scrapyd-webapp are comparing it to the libraries listed below
- Restrict crawl and scraping scope using matchers.☆25Updated 8 years ago
- Python implementation of the Parsley language for extracting structured data from web pages☆92Updated 7 years ago
- Library designed to replace the SQLite backend by a MongoDB backend on Scrapy queue management☆17Updated 7 years ago
- [UNMAINTAINED] Deploy, run and monitor your Scrapy spiders.☆11Updated 9 years ago
- Python scripts for scraping bus ticket data from the websites of BoltBus, Greyhound, Megabus, GoBus, Amtrak, Peterpan, and EasternTravel.☆39Updated 4 years ago
- Find which links on a web page are pagination links☆29Updated 8 years ago
- A Python library for extracting titles, images, descriptions and canonical urls from HTML.☆147Updated 4 years ago
- Simple Web UI for Scrapy spider management via Scrapyd☆51Updated 6 years ago
- Monitors prices of Amazon products via Product Advertising API☆156Updated 5 years ago
- Dynamic data analysis over the web. The logic to your data dashboards.☆156Updated 9 years ago
- Scrapy Middleware to set a random User-Agent for every Request.☆201Updated 5 years ago
- A component that tries to avoid downloading duplicate content☆27Updated 6 years ago
- Scrapy spider middleware to split an item into multiple items using a multi-valued key☆20Updated 8 years ago
- ☆143Updated 9 years ago
- "Scrape Easy" - an extension of the Scrapy framework.☆188Updated 8 years ago
- Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations☆40Updated 8 months ago
- Some scrapy and web.py exmaples☆79Updated 7 years ago
- Python module to scrape Amazon reviews.☆25Updated 12 years ago
- 📑⚙️ Python/Django reference implementation of the ERAV data model☆21Updated 5 years ago
- Gevent Crawling in Python, with Utilities☆23Updated 9 years ago
- [UNMAINTAINED] Firefox addon for Scrapely☆5Updated 9 years ago
- Small set of utilities to simplify writing Scrapy spiders.☆49Updated 9 years ago
- Python library that uses selenium and phantomjs to automate Facebook Group administration☆27Updated 6 years ago
- Detect and classify pagination links☆15Updated 4 years ago
- Scrape the Google search result with Scrapy.☆98Updated 4 years ago
- Sample projects showcasing Scrapinghub tech☆138Updated last year
- Amazon Price scraping tool☆37Updated 11 years ago
- A dashboard for rq-scheduler jobs☆17Updated 2 years ago
- ☆20Updated 11 years ago
- bookmark management for the Django web framework☆17Updated 8 years ago