Alternatives and similar repositories for crawl-anywhere:

Users that are interested in crawl-anywhere are comparing it to the libraries listed below

• spinscale / elasticsearch-opennlp-plugin
  Additional opennlp mapping type for elasticsearch in order to perform named entity recognition
  ☆136Updated 8 years ago
• commoncrawl / commoncrawl-crawler
  The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)
  ☆213Updated 2 years ago
• Smerity / cc-warc-examples
  CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop
  ☆56Updated 3 years ago
• momer / nutch-selenium
  ☆28Updated 8 years ago
• seagatesoft / sde
  Structured Data Extractor. An application to extract structured data from web pages. It uses Data Extraction Based on Partial Tree Alignm…
  ☆49Updated 12 years ago
• kojisekig / KEA-lucene
  ☆18Updated 8 years ago
• Norconex / crawlers
  Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or fi…
  ☆187Updated this week
• scrapinghub / mdr
  A python library detect and extract listing data from HTML page.
  ☆108Updated 7 years ago
• pramitchoudhary / TubeKit
  crawler for YouTube
  ☆48Updated 11 years ago
• hollingsworthd / ScreenSlicer
  Automatic, zero-config web scraping -- written in Java, has no dependency on Java EE or app servers, and the web scraper has a restful/JS…
  ☆155Updated 7 years ago
• shirk3y / selectorgadget-plus
  A tool for manage website extraction configs
  ☆37Updated 11 years ago
• grafos-ml / frappe
  Recommendations Serving Engine using python
  ☆28Updated 9 years ago
• mohaps / tldrzr
  Algorithmic summarizer for RSS/Atom Feeds, Web Urls and arbitrary text. Codebase for the application deployed at http://tldrzr.herokuapp.…
  ☆53Updated 8 years ago
• BayanGroup / nutch-custom-search
  ☆66Updated 8 years ago
• CottageLabs / facetview2
  a pure javascript frontend for ElasticSearch search indices.
  ☆79Updated 7 years ago
• LinkedInAttic / rest.li-api-hub
  API Hub is a web UI for browsing and searching a catalog of Rest.li APIs.
  ☆74Updated 5 years ago
• nytlabs / hive
  A platform for backing crowdsourcing websites, built in golang for elasticsearch
  ☆360Updated 4 years ago
• openplans / openblock
  OpenBlock is a web application and RESTful service that allows users to browse and search their local area for "hyper-local news
  ☆61Updated 3 years ago
• FCC / Crawler
  Crawler is a bare-bones spider designed to quickly and effectively build an index of all files and pages on a given Web site as well as t…
  ☆91Updated 12 years ago
• openpreserve / pagelyzer
  Suite of tools for detecting changes in web pages and their rendering
  ☆54Updated last year
• PerfectMemory / freebase-api
  FreebaseAPI is a library to use the Freebase API (data mapper + low level API)
  ☆42Updated 10 years ago
• endgameinc / elasticsearch-term-plugin
  Term List Matching Plugin for ElasticSearch
  ☆26Updated 11 years ago
• flaxsearch / solr-es-comparison
  ☆18Updated 9 years ago
• germanattanasio / personalized-recommendations
  Personalized Recommendations helps you find the best place to go on vacations by using the Concept Insights and Tradeoff Analytics Watso…
  ☆16Updated 9 years ago
• arunkjn / screenshot-service
  Screenshot as a service with phantomJS headless browser
  ☆58Updated 11 years ago
• ericabouaf / pipes2js
  Compile Yahoo! Pipes to Javascript (Node.js)
  ☆44Updated 12 years ago
• Wikidata / primarysources
  Approve or reject statements from third-party datasets
  ☆146Updated 6 years ago
• scrapy / slybot
  ☆223Updated 9 years ago
• deezer / weslang
  A language detection Web Service
  ☆53Updated 7 years ago
• graphaware / recommendations-meetup
  Skeleton for Meetup - Building your own recommendation engine in an hour
  ☆29Updated 3 years ago