Alternatives and similar repositories for cc-mrjob:

Users that are interested in cc-mrjob are comparing it to the libraries listed below

• trivio / common_crawl_index
  Index URLs in Common Crawl
  ☆194Updated 7 years ago
• ikreymer / cdx-index-client
  A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/
  ☆182Updated 6 years ago
• scrapinghub / webstruct
  NER toolkit for HTML data
  ☆257Updated 8 months ago
• scrapinghub / aile
  Automatic Item List Extraction
  ☆87Updated 8 years ago
• commoncrawl / cc-index-table
  Index Common Crawl archives in tabular format
  ☆110Updated 2 months ago
• dat / pyner
  Python interface to the Stanford Named Entity Recognizer
  ☆291Updated 3 years ago
• GregBowyer / cld2-cffi
  Python bindings to the Compact Language Detector
  ☆33Updated 4 years ago
• nickdavidhaynes / spacy-cld
  Language detection extension for spaCy 2.0+
  ☆112Updated 5 years ago
• vu3jej / scrapy-corenlp
  ☆59Updated 3 years ago
• turian / topia.termextract
  Updates to Zope's keyphrase extractor (forked from 1.1.0)
  ☆66Updated 7 years ago
• explosion / displacy-ent
  displaCy-ent.js: An open-source named entity visualiser for the modern web
  ☆198Updated 6 years ago
• internetarchive / warc
  Python library for reading and writing warc files
  ☆239Updated 2 years ago
• TeamHG-Memex / deep-deep
  Adaptive crawler which uses Reinforcement Learning methods
  ☆169Updated 6 years ago
• tokestermw / spacy_hunspell
  Hunspell extension for spaCy 2.0.
  ☆94Updated 5 months ago
• pydepta / pydepta
  A python implementation of DEPTA
  ☆83Updated 8 years ago
• mediacloud / date_guesser
  A library to extract a publication date from a web page, along with a measure of the accuracy.
  ☆42Updated 5 years ago
• commoncrawl / cc-pyspark
  Process Common Crawl data with Python and Spark
  ☆411Updated last month
• davidmogar / cucco
  Text normalization library for Python
  ☆204Updated 6 years ago
• cocrawler / cdx_toolkit
  A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine
  ☆166Updated 3 weeks ago
• TeamHG-Memex / Formasaurus
  Formasaurus tells you the type of an HTML form and its fields using machine learning
  ☆118Updated 7 months ago
• edsu / microdata
  python library for extracting html microdata
  ☆166Updated last year
• TeamHG-Memex / html-text
  Extract text from HTML
  ☆133Updated 4 years ago
• thavelick / summarize
  A python library for simple text summarization
  ☆218Updated 9 years ago
• scrapinghub / mdr
  A python library detect and extract listing data from HTML page.
  ☆109Updated 7 years ago
• webrecorder / warcio
  Streaming WARC/ARC library for fast web archive IO
  ☆395Updated last month
• snowballstem / pystemmer
  Python stemming library using snowball stemmers
  ☆248Updated 3 months ago
• TeamHG-Memex / scrapy-dockerhub
  [UNMAINTAINED] Deploy, run and monitor your Scrapy spiders.
  ☆11Updated 9 years ago
• mitll / topic-clustering
  ☆43Updated 9 years ago
• pudo / normality
  A tiny library for Python text normalisation. Useful for ad-hoc text processing.
  ☆146Updated 3 weeks ago
• explosion / spacy-dev-resources
  💫 Scripts, tools and resources for developing spaCy
  ☆125Updated 5 years ago