Alternatives and similar repositories for wikiextractor

Users that are interested in wikiextractor are comparing it to the libraries listed below

• MorinoseiMorizo / jparacrawl-finetune
  An example usage of JParaCrawl pre-trained Neural Machine Translation (NMT) models.
  ☆105Updated 4 years ago
• tsuruoka-lab / BSD
  The Business Scene Dialogue corpus
  ☆73Updated 4 years ago
• rpryzant / JESC
  A large parallel corpus of English and Japanese
  ☆87Updated 8 years ago
• nttcslab-nlp / word_align
  A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT
  ☆27Updated 5 years ago
• octanove / shiba
  Pytorch implementation and pre-trained Japanese model for CANINE, the efficient character-level transformer.
  ☆89Updated 2 years ago
• NathanDuran / Switchboard-Corpus
  Utilities for Processing the Switchboard Dialogue Act Corpus
  ☆73Updated 5 years ago
• singletongue / japanese-bert
  BERT models with tokenization for Japanese texts.
  ☆14Updated 6 years ago
• teaspn / teaspn-server
  A sample implementation of the TEASPN server
  ☆19Updated 6 years ago
• nttcslab-nlp / spanalign
  SpanAlign: Sentence Alignment Method based on Cross-Language Span Prediction and ILP
  ☆14Updated 4 years ago
• megagonlabs / t5-japanese
  Codes to pre-train Japanese T5 models
  ☆40Updated 4 years ago
• mynlp / niilc-qa
  NIILC QA data
  ☆18Updated 10 years ago
• M4t1ss / SoftAlignments
  Neural macine translation soft alignment visualisations for web and command line
  ☆72Updated 4 years ago
• snukky / wikiedits
  Automatic extraction of edited sentences from text edition histories.
  ☆83Updated 3 years ago
• M4t1ss / parallel-corpora-tools
  Tools for filtering and cleaning parallel and monolingual corpora for machine translation and other natural language processing tasks.
  ☆41Updated 2 years ago
• AkariAsai / extractive_rc_by_runtime_mt
  Code and datasets of "Multilingual Extractive Reading Comprehension by Runtime Machine Translation"
  ☆40Updated 7 years ago
• ufal / multilexnorm2021
  MultiLexNorm 2021 competition system from ÚFAL
  ☆15Updated 4 years ago
• ucam-smt / sgnmt
  Decoding platform for machine translation research
  ☆54Updated 6 years ago
• tsuruoka-lab / AMI-Meeting-Parallel-Corpus
  AMI Meeting Parallel Corpus
  ☆11Updated 5 years ago
• nishkalavallabhi / OneStopEnglishCorpus
  Repository for Vajjala & Lucic (2018)
  ☆67Updated last year
• SkelterLabsInc / JaQuAD
  JaQuAD: Japanese Question Answering Dataset for Machine Reading Comprehension (2022, Skelter Labs)
  ☆108Updated 3 years ago
• ku-nlp / kyoto-reader
  A processor for KyotoCorpus, KWDLC, and AnnotatedFKCCorpus
  ☆10Updated last year
• odashi / small_parallel_enja
  50k English-Japanese Parallel Corpus for Machine Translation Benchmark.
  ☆98Updated 6 years ago
• EdinburghNLP / opus-100-corpus
  ☆94Updated 2 years ago
• Hyperparticle / udify
  A single model that parses Universal Dependencies across 75 languages. Given a sentence, jointly predicts part-of-speech tags, morphology…
  ☆225Updated 3 years ago
• laboroai / Laboro-ParaCorpus
  Scripts for creating a Japanese-English parallel corpus and training NMT models
  ☆18Updated 4 years ago
• shyyhs / CourseraParallelCorpusMining
  Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation
  ☆15Updated last year
• Mao-KU / JASS
  JASS: Japanese-specific Sequence to Sequence Pre-training for Neural Machine Translation (LREC2020) & Linguistically Driven Multi-Task Pr…
  ☆16Updated 4 years ago
• robertostling / efmaral
  Efficient Markov Chain word alignment
  ☆52Updated 4 years ago
• tomo-wb / Lang8-NAIST-extractor
  ☆30Updated 5 years ago
• ku-nlp / KWDLC
  Kyoto University Web Document Leads Corpus
  ☆83Updated 2 years ago