Alternatives and similar repositories for wikiextractor

Users that are interested in wikiextractor are comparing it to the libraries listed below

• MorinoseiMorizo / jparacrawl-finetune
  An example usage of JParaCrawl pre-trained Neural Machine Translation (NMT) models.
  ☆103Updated 4 years ago
• tsuruoka-lab / BSD
  The Business Scene Dialogue corpus
  ☆70Updated 3 years ago
• octanove / shiba
  Pytorch implementation and pre-trained Japanese model for CANINE, the efficient character-level transformer.
  ☆89Updated last year
• rpryzant / JESC
  A large parallel corpus of English and Japanese
  ☆84Updated 7 years ago
• nttcslab-nlp / word_align
  A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT
  ☆27Updated 4 years ago
• nttcslab-nlp / spanalign
  SpanAlign: Sentence Alignment Method based on Cross-Language Span Prediction and ILP
  ☆14Updated 4 years ago
• nishkalavallabhi / OneStopEnglishCorpus
  Repository for Vajjala & Lucic (2018)
  ☆65Updated last year
• Hyperparticle / udify
  A single model that parses Universal Dependencies across 75 languages. Given a sentence, jointly predicts part-of-speech tags, morphology…
  ☆223Updated 2 years ago
• megagonlabs / t5-japanese
  Codes to pre-train Japanese T5 models
  ☆40Updated 4 years ago
• singletongue / japanese-bert
  BERT models with tokenization for Japanese texts.
  ☆14Updated 5 years ago
• mynlp / niilc-qa
  NIILC QA data
  ☆18Updated 9 years ago
• tsuruoka-lab / AMI-Meeting-Parallel-Corpus
  AMI Meeting Parallel Corpus
  ☆11Updated 4 years ago
• awslabs / mlm-scoring
  Python library & examples for Masked Language Model Scoring (ACL 2020)
  ☆347Updated 2 years ago
• naist-nlp / mbrs
  A library for minimum Bayes risk (MBR) decoding
  ☆47Updated 3 months ago
• soaxelbrooke / python-bpe
  Byte Pair Encoding for Python!
  ☆231Updated 3 years ago
• octanove / grammartagger
  GrammarTagger — A Neural Multilingual Grammar Profiler for Language Learning
  ☆30Updated 4 years ago
• NathanDuran / Switchboard-Corpus
  Utilities for Processing the Switchboard Dialogue Act Corpus
  ☆70Updated 4 years ago
• ufal / multilexnorm2021
  MultiLexNorm 2021 competition system from ÚFAL
  ☆15Updated 3 years ago
• bitextor / bicleaner
  Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.
  ☆158Updated last year
• odashi / small_parallel_enja
  50k English-Japanese Parallel Corpus for Machine Translation Benchmark.
  ☆96Updated 6 years ago
• ku-nlp / KyotoCorpus
  Kyoto University Text Corpus
  ☆66Updated 2 years ago
• alinear-corp / albert-japanese
  BERT with SentencePiece for Japanese text.
  ☆33Updated 3 years ago
• M4t1ss / parallel-corpora-tools
  Tools for filtering and cleaning parallel and monolingual corpora for machine translation and other natural language processing tasks.
  ☆41Updated last year
• ucam-smt / sgnmt
  Decoding platform for machine translation research
  ☆55Updated 6 years ago
• AkariAsai / extractive_rc_by_runtime_mt
  Code and datasets of "Multilingual Extractive Reading Comprehension by Runtime Machine Translation"
  ☆40Updated 6 years ago
• uds-lsv / bert-stable-fine-tuning
  On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
  ☆137Updated 2 years ago
• ku-nlp / kyoto-reader
  A processor for KyotoCorpus, KWDLC, and AnnotatedFKCCorpus
  ☆10Updated last year
• Helsinki-NLP / OpusFilter
  OpusFilter - Parallel corpus processing toolkit
  ☆109Updated last month
• snukky / wikiedits
  Automatic extraction of edited sentences from text edition histories.
  ☆83Updated 3 years ago
• ku-nlp / KWDLC
  Kyoto University Web Document Leads Corpus
  ☆83Updated last year