Alternatives and similar repositories for common_crawl_corpus

Users that are interested in common_crawl_corpus are comparing it to the libraries listed below

• transducens / linguacrawl
  Crawling engine that crawls a set of top-level domains looking for documents in a list of languages
  ☆11Updated last year
• mediacloud / date_guesser
  A library to extract a publication date from a web page, along with a measure of the accuracy.
  ☆41Updated 5 years ago
• anlausch / ArguminSci
  Analyze Argumentation and Rhetorical Aspects in Scientific Writing.
  ☆19Updated 2 years ago
• BramVanroy / spacy-extreme
  An example of how to use spaCy for extremely large files without running into memory issues
  ☆36Updated 2 years ago
• kensho-technologies / qwikidata
  Python tools for interacting with Wikidata
  ☆154Updated last year
• jacklxc / ScientificDiscourseTagging
  Implementation for EACL 2021 paper "Scientific Discourse Tagging for Evidence Extraction".
  ☆20Updated 3 years ago
• dataiku / dss-plugin-nlp-preparation
  Dataiku DSS plugin to detect languages, correct misspellings, and clean text data 🧼
  ☆22Updated 5 months ago
• bltlab / paranames
  ParaNames: A multilingual resource for parallel names
  ☆34Updated last year
• andreasvc / readability
  Measure the readability of a given text using surface characteristics
  ☆79Updated 5 months ago
• UCREL / Multilingual-USAS
  Lexicons for the Multilingual UCREL Semantic Analysis System
  ☆43Updated last year
• racai-ai / pyeurovoc
  Legal document classification with EuroVoc descriptors on 22 languages.
  ☆26Updated 2 years ago
• kermitt2 / grobid-ner
  A Named-Entity Recogniser based on Grobid.
  ☆55Updated 2 months ago
• dkpro / dkpro-c4corpus
  DKPro C4CorpusTools is a collection of tools for processing CommonCrawl corpus, including Creative Commons license detection, boilerplate…
  ☆52Updated 5 years ago
• talmago / spacy_crfsuite
  sequence tagging with spaCy and crfsuite
  ☆20Updated 2 years ago
• jbesomi / fastlaw
  Fastlaw's purpose is to replace generic word embeddings for work on supervised machine learning NLP-tasks with legal texts.
  ☆38Updated 6 years ago
• revuel / PatternOmatic
  Finds linguistic patterns effortlessly
  ☆37Updated last year
• stressosaurus / raw-data-google-ngram
  This will download and process the Google Ngram data.
  ☆19Updated 2 years ago
• tokestermw / spacy_grammar
  Language Tool style grammar handling with spaCy 2.0
  ☆42Updated 6 years ago
• TurkuNLP / wikibert
  BERT models for many languages created from Wikipedia texts
  ☆33Updated 5 years ago
• cisnlp / GlotWeb
  🕸 GlotWeb: Web Indexing for Low-Resource Languages -- under construction.
  ☆13Updated 3 months ago
• TextRazor / textrazor-python
  Python SDK for the TextRazor Text Analytics API
  ☆20Updated last year
• stanfordnlp / ColBERT-QA
  Code for Relevance-guided Supervision for OpenQA with ColBERT (TACL'21)
  ☆41Updated 3 years ago
• stefan-it / gc4lm
  GC4LM: A Colossal (Biased) language model for German
  ☆13Updated 4 years ago
• JacobTyo / Valla
  ☆28Updated 2 years ago
• ICLRandD / Case2Vec
  A simple web application for searching Word2Vec embeddings derived from approximately 2,000 law reports published by the The Incorporated…
  ☆25Updated 2 years ago
• pmbaumgartner / spacy-setfit-textcat
  ☆30Updated 3 years ago
• alexyorke / butter-fingers
  A python library to generate highly realistic typos (fuzz-testing)
  ☆12Updated 4 months ago
• mpsilfve / finer-data
  ☆25Updated 5 years ago
• google-research-datasets / TF-IDF-IIF-top100-wordlists
  These are lists for a variety of languages containing words that are distinctive to each language.
  ☆38Updated 3 years ago
• bjascob / pyInflect
  A python module for word inflections designed for use with spaCy.
  ☆92Updated 5 years ago