Alternatives and similar repositories for crawl-to-the-future:

Users that are interested in crawl-to-the-future are comparing it to the libraries listed below

• trec-kba / streamcorpus
  common data interchange format for document processing pipelines that apply natural language processing tools to large streams of text
  ☆34Updated 8 years ago
• seomoz / mltk
  mltk - Moz Language Tool Kit
  ☆12Updated 9 years ago
• Parsely / schemato
  Modularly extensible semantic metadata validator
  ☆83Updated 9 years ago
• nytlabs / linguo
  a set of services that provide NLP facilities
  ☆25Updated 4 years ago
• pudo-attic / archivekit
  ArchiveKit manages data and documents during ETL processes, either on a local file system or on S3.
  ☆15Updated 9 years ago
• ShinNoNoir / twitterwebsearch
  (BROKEN, help wanted)
  ☆15Updated 8 years ago
• xtannier / WebAnnotator
  WebAnnotator is a tool for annotating Web pages. WebAnnotator is implemented as a Firefox extension (https://addons.mozilla.org/en-US/fi…
  ☆48Updated 3 years ago
• scrapinghub / page_finder
  Find which links on a web page are pagination links
  ☆29Updated 8 years ago
• willf / segment
  A tool to segment text based on frequencies and the Viterbi algorithm "#TheBoyWhoLived" => ['#', 'The', 'Boy', 'Who', 'Lived']
  ☆82Updated 8 years ago
• frnsys / broca
  rapid nlp prototyping
  ☆72Updated 2 years ago
• jimmycallin / plainstream
  Sometimes you just need a lot of text. Plainstream is a small Python app that provides you with a plain text stream directly from Wikiped…
  ☆24Updated last year
• commonsearch / gumbocy
  Python binding for gumbo-parser using Cython
  ☆14Updated 8 years ago
• istresearch / traptor
  Traptor -- A distributed Twitter feed
  ☆26Updated 2 years ago
• rmax / databrewer
  The missing datasets manager. Like hombrew but for datasets. CLI-tool for search and discover datasets!
  ☆41Updated 7 years ago
• seomoz / mozsci
  Data science tools from Moz
  ☆22Updated 8 years ago
• TeamHG-Memex / scrapy-dockerhub
  [UNMAINTAINED] Deploy, run and monitor your Scrapy spiders.
  ☆11Updated 9 years ago
• MartinPaulEve / PlotSummary
  Plots various graphs for a series of plaintext files in a directory
  ☆19Updated 8 years ago
• pudo-attic / scrapekit
  Python library with common functionality for writing web scrapers
  ☆102Updated 9 years ago
• turian / topia.termextract
  Updates to Zope's keyphrase extractor (forked from 1.1.0)
  ☆66Updated 7 years ago
• frictionlessdata / schemas
  [DEPRECATED] Please use https://github.com/frictionlessdata/specs
  ☆17Updated 7 years ago
• dsoprea / TightOCR
  A slim, non-SWIG Python adapter to CTesseract (Tesseract OCR for C).
  ☆24Updated 10 years ago
• anthonygarvan / MinHash
  ☆24Updated 6 years ago
• LuminosoInsight / assoc-space
  Compute association strength over semantic networks in a dimensionality-reduced form.
  ☆33Updated 9 years ago
• commoncrawl / gzipstream
  gzipstream allows Python to process multi-part gzip files from a streaming source
  ☆23Updated 7 years ago
• datalib / StatsCounter
  Python's missing statistical Swiss Army knife
  ☆15Updated 9 years ago
• GregBowyer / cld2-cffi
  Python bindings to the Compact Language Detector
  ☆33Updated 4 years ago
• TeamHG-Memex / MaybeDont
  A component that tries to avoid downloading duplicate content
  ☆27Updated 6 years ago
• TeamHG-Memex / extract-html-diff
  extract difference between two html pages
  ☆32Updated 6 years ago
• seomoz / vocab
  Vocabulary using n-grams
  ☆16Updated 6 years ago
• paulsmith / templatemaker
  templatemaker is a Python library that can extract data from files with a similar format, like HTML pages.
  ☆63Updated 4 years ago