atupal / ccrawler
A distrubuted crawler ues celery.
☆17Updated 10 years ago
Alternatives and similar repositories for ccrawler
Users that are interested in ccrawler are comparing it to the libraries listed below
Sorting:
- Search engine base (crawler, indexer and parser) using Python, Celery, RabbitMQ, CouchDB and Whoosh.☆11Updated last year
- Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations☆40Updated 11 months ago
- Tornado Web Crawler☆66Updated 12 years ago
- ☆32Updated last year
- A Django based search engine powered by CouchDB, celery and whoosh.☆49Updated 9 years ago
- [not actively maintained] The C++ webkit-server from capybara-webkit with useful extensions and Python bindings☆48Updated 4 years ago
- Jabba's headless webkit browser for scraping AJAX-powered webpages.☆92Updated 10 years ago
- Smartyparse: A python library for smart dynamic binary de/encoding.☆15Updated 7 years ago
- Turn your IPython console into a cross-database SQL client☆31Updated 8 years ago
- Python library to make it easy to upsert on MySQL, PostgreSQL, and SQLite3.☆18Updated 2 years ago
- Tool to flatten stream of JSON-like objects, configured via schema☆33Updated 5 years ago
- Scrapy pipeline which allows you to store scrapy items in appery.io database.☆14Updated 7 years ago
- phantomjs based screenshot server☆18Updated 10 years ago
- In browser VNC client through websockets☆30Updated 13 years ago
- A scrapy extension to store requests and responses information in storage service☆26Updated 3 years ago
- Scan for missing timeout calls in python source files☆18Updated 5 years ago
- Restrict crawl and scraping scope using matchers.☆25Updated 8 years ago
- Scraping bhinneka.com, just for fun☆14Updated 12 years ago
- Kel cluster management Python library☆10Updated 7 years ago
- Small set of utilities to simplify writing Scrapy spiders.☆49Updated 9 years ago
- A Python module to fetch and parse results from different search engines.☆77Updated 6 years ago
- Gevent Crawling in Python, with Utilities☆23Updated 10 years ago
- Python Dictionaries and Lists on Steroids☆23Updated 10 years ago
- Scrapy middleware which allows to crawl only new content☆80Updated 2 years ago
- python library for interacting with SolrCloud☆36Updated 4 years ago
- ☆18Updated 8 years ago
- Python SMTP client and Email for Humans™☆82Updated 6 years ago
- Integrates terminado (a web based terminal) with flask☆15Updated 7 years ago
- Fast Python Bloom Filter using Mmap☆13Updated 12 years ago
- Builds HTML from Python (recovered from local installation since original was deleted)☆11Updated 2 years ago