scrapinghub / scaws
Extensions for using Scrapy on Amazon AWS
☆32Updated 11 years ago
Related projects ⓘ
Alternatives and complementary repositories for scaws
- MongoDB extensions for Scrapy☆44Updated 10 years ago
- Small set of utilities to simplify writing Scrapy spiders.☆49Updated 9 years ago
- Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations☆40Updated 6 months ago
- Fast Python Bloom Filter using Mmap☆13Updated 12 years ago
- Restrict crawl and scraping scope using matchers.☆25Updated 8 years ago
- Paginating the web☆37Updated 10 years ago
- Find which links on a web page are pagination links☆29Updated 7 years ago
- Crawlera tools☆26Updated 8 years ago
- ☆32Updated 10 months ago
- Python library that uses selenium and phantomjs to automate Facebook Group administration☆28Updated 6 years ago
- A scrapy pipeline which send items to Elastic Search server☆98Updated 6 years ago
- Detect and classify pagination links☆14Updated 4 years ago
- ☆49Updated 2 years ago
- Traptor -- A distributed Twitter feed☆26Updated 2 years ago
- Running scrapy spider programmatically.☆47Updated 8 years ago
- Simple program that summarize text.☆10Updated 14 years ago
- Discover repositories you should be following on Github.☆30Updated 12 years ago
- Site Hound (previously THH) is a Domain Discovery Tool☆23Updated 3 years ago
- Gevent Crawling in Python, with Utilities☆23Updated 9 years ago
- Block certain IP addresses from accessing your Flask app☆16Updated 6 years ago
- Simple Web UI for Scrapy spider management via Scrapyd☆50Updated 6 years ago
- Scrapy pipeline which allows you to store scrapy items in a solr server.☆19Updated 8 years ago
- High Level Kafka Scanner☆19Updated 7 years ago
- Scrapy schema validation pipeline and Item builder using JSON Schema☆44Updated 3 years ago
- A pastebin for tables.☆34Updated 11 years ago
- Slides to learn a little natural language processing (NLP) with Python. Written in reST with S5/Docutils.☆28Updated 12 years ago
- Python video summarization. Visit the public API at -- www.shorten.tv (EDIT: The domain expired and youtube blocked it ..)☆81Updated 2 years ago
- Software stack used to run Portia spiders in Scrapinghub cloud☆10Updated 5 years ago
- A Python client for Chrome's DevTools protocol / a headless chrome control library☆15Updated 6 years ago
- A distributed system for mining common crawl using SQS, AWS-EC2 and S3☆14Updated 10 years ago