atupal / ccrawler
A distrubuted crawler ues celery.
☆17Updated 10 years ago
Alternatives and similar repositories for ccrawler:
Users that are interested in ccrawler are comparing it to the libraries listed below
- Search engine base (crawler, indexer and parser) using Python, Celery, RabbitMQ, CouchDB and Whoosh.☆11Updated last year
- Find which links on a web page are pagination links☆29Updated 8 years ago
- Crawlera tools☆26Updated 9 years ago
- Fast Python Bloom Filter using Mmap☆13Updated 12 years ago
- ☆18Updated 8 years ago
- Tornado Web Crawler☆66Updated 12 years ago
- Toy web crawler☆21Updated 13 years ago
- Simple Web UI for Scrapy spider management via Scrapyd☆51Updated 6 years ago
- Restrict crawl and scraping scope using matchers.☆25Updated 8 years ago
- Bringing sanity to world of messed-up data☆66Updated 10 years ago
- Scrapy pipeline which allows you to store scrapy items in appery.io database.☆14Updated 7 years ago
- Python Domain Specific Language Tools☆84Updated 2 years ago
- Smartyparse: A python library for smart dynamic binary de/encoding.☆15Updated 7 years ago
- Python bindings for CLD2.☆16Updated 6 years ago
- Kel cluster management Python library☆10Updated 6 years ago
- Small set of utilities to simplify writing Scrapy spiders.☆49Updated 9 years ago
- Webkit based scriptable web browser for python.☆29Updated 12 years ago
- Tool to flatten stream of JSON-like objects, configured via schema☆33Updated 5 years ago
- Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations☆40Updated 10 months ago
- Jabba's headless webkit browser for scraping AJAX-powered webpages.☆91Updated 10 years ago
- dank key/value store high-level APIs☆18Updated 7 years ago
- python library for interacting with SolrCloud☆36Updated 4 years ago
- Scan for missing timeout calls in python source files☆18Updated 5 years ago
- A Python client for Chrome's DevTools protocol / a headless chrome control library☆15Updated 6 years ago
- ☆32Updated last year
- Collection of Scrapy utilities (extensions, middlewares, pipelines, etc)☆32Updated 7 years ago
- Pure Python XPath implementation that operates on ElementTree objects.☆11Updated 10 years ago
- Integrates terminado (a web based terminal) with flask☆15Updated 7 years ago
- Turn your IPython console into a cross-database SQL client☆31Updated 8 years ago
- Python & Numpy embedded within C++ using Cython as magic glue☆12Updated 11 years ago