scalingexcellence / scrapy-solr
Scrapy pipeline which allows you to store scrapy items in a solr server.
☆19Updated 8 years ago
Alternatives and similar repositories for scrapy-solr:
Users that are interested in scrapy-solr are comparing it to the libraries listed below
- Restrict crawl and scraping scope using matchers.☆25Updated 8 years ago
- An online sentiment analyzer built with Flask and TextBlob☆15Updated 11 years ago
- This is a REST Server endpoint built using Flask and Python.☆23Updated 2 years ago
- Virtual patent marking crawler at iproduct.epfl.ch☆14Updated 7 years ago
- Find which links on a web page are pagination links☆29Updated 8 years ago
- Site Hound (previously THH) is a Domain Discovery Tool☆23Updated 3 years ago
- Traptor -- A distributed Twitter feed☆26Updated 2 years ago
- Library designed to replace the SQLite backend by a MongoDB backend on Scrapy queue management☆17Updated 7 years ago
- Automatic tagging and analysis of documents in an Apache Solr index for faceted search by RDF(S) Ontologies & SKOS thesauri☆47Updated 3 years ago
- Named-Entity Recognition extension for Google Refine / OpenRefine☆72Updated 7 years ago
- common data interchange format for document processing pipelines that apply natural language processing tools to large streams of text☆35Updated 8 years ago
- WebAnnotator is a tool for annotating Web pages. WebAnnotator is implemented as a Firefox extension (https://addons.mozilla.org/en-US/fi…☆48Updated 3 years ago
- Demo of the Newspaper article extraction library.☆29Updated 10 years ago
- Paginating the web☆37Updated 11 years ago
- Pipeline for distributed Natural Language Processing, made in Python☆65Updated 8 years ago
- Keyword query search engine on semantic store/linked data web☆9Updated 9 years ago
- A python implementation of DEPTA☆83Updated 8 years ago
- Easy extraction of keywords and engines from search engine results pages (SERPs).☆90Updated 3 years ago
- Scrape every LinkedIn public profile using Scrapy (Python)☆15Updated 10 years ago
- Simple type converters: make ints, floats, bools and dates from your strings!☆11Updated 8 years ago
- Search engine base (crawler, indexer and parser) using Python, Celery, RabbitMQ, CouchDB and Whoosh.☆11Updated last year
- Python package to detect and return RSS / Atom feeds for a given website. The tool supports major blogging platform including Wordpress, …☆21Updated 3 years ago
- ArchiveKit manages data and documents during ETL processes, either on a local file system or on S3.☆15Updated 9 years ago
- A Python library for extracting titles, images, descriptions and canonical urls from HTML.☆148Updated 4 years ago
- A python library detect and extract listing data from HTML page.☆108Updated 7 years ago
- Python/Django based webapps and web user interfaces for search, structure (meta data management like thesaurus, ontologies, annotations a…☆96Updated 2 years ago
- Scraper built with Scrapy.☆15Updated 7 months ago
- extract relationships from standardized terms from corpus of interest with deep learning☆20Updated 5 years ago
- Keyword Extraction system using Brown Clustering - (This version is trained to extract keywords from job listings)☆17Updated 10 years ago
- A web application for collecting tweets from the twitter API☆15Updated 10 years ago