atupal / ccrawlerLinks
A distrubuted crawler ues celery.
☆16Updated 10 years ago
Alternatives and similar repositories for ccrawler
Users that are interested in ccrawler are comparing it to the libraries listed below
Sorting:
- Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations☆40Updated last year
- A component that tries to avoid downloading duplicate content☆27Updated 7 years ago
- python library for interacting with SolrCloud☆36Updated 4 years ago
- Tornado Web Crawler☆66Updated 12 years ago
- A Python wrapper for MADlib(http://madlib.net) - an open source library for scalable in-database machine learning algorithms☆63Updated 4 years ago
- Jabba's headless webkit browser for scraping AJAX-powered webpages.☆91Updated 10 years ago
- Python SMTP client and Email for Humans™☆82Updated 6 years ago
- Scrapy middleware for the autologin☆36Updated 7 years ago
- Cubes OLAP Examples☆74Updated 7 years ago
- Turn your IPython console into a cross-database SQL client☆31Updated 9 years ago
- Search engine base (crawler, indexer and parser) using Python, Celery, RabbitMQ, CouchDB and Whoosh.☆11Updated last month
- Data analysis tool.☆85Updated 2 years ago
- A Python module to fetch and parse results from different search engines.☆77Updated 6 years ago
- Application Driven Stats Monitoring☆231Updated 10 years ago
- Web API Authentication with SSH Public Keys☆161Updated 12 years ago
- Find which links on a web page are pagination links☆29Updated 8 years ago
- Python Domain Specific Language Tools☆84Updated 3 years ago
- extract difference between two html pages☆32Updated 7 years ago
- Python library with common functionality for writing web scrapers☆102Updated 10 years ago
- [not actively maintained] The C++ webkit-server from capybara-webkit with useful extensions and Python bindings☆48Updated 4 years ago
- Convert URL's to a normalized unicode format☆67Updated 7 years ago
- iCQA - Intelligent Community Question Answering Framework☆31Updated 8 years ago
- Python Dictionaries and Lists on Steroids☆23Updated 10 years ago
- Supervisor On/Off: an alternative web interface for supervisor☆50Updated 8 years ago
- Builds HTML from Python (recovered from local installation since original was deleted)☆11Updated 2 years ago
- Rocket is a Pure-Python multi-threaded server.☆20Updated 6 years ago
- Supervisor dashboard☆49Updated 9 years ago
- Site Hound (previously THH) is a Domain Discovery Tool☆23Updated 4 years ago
- A cron job runner with self-healing and job dependencies.☆134Updated 3 years ago
- A project to attempt to automatically login to a website given a single seed☆125Updated 2 years ago