Alternatives and similar repositories for nlp-datasets

Users that are interested in nlp-datasets are comparing it to the libraries listed below

• IndoNLP / nusa-writes
  NusaWrites is an in-depth analysis of corpora collection strategy and a comprehensive language modeling benchmark for underrepresented an…
  ☆27Updated last year
• shyyhs / CourseraParallelCorpusMining
  Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation
  ☆15Updated last year
• tatHi / optok
  ☆11Updated 4 years ago
• lgessler / microbert
  A tiny BERT for low-resource monolingual models
  ☆31Updated 2 months ago
• gunnxx / indonesian-mt-data
  Benchmarking Multidomain English-Indonesian Machine Translation
  ☆16Updated 4 years ago
• shisa-ai / shaberi
  Lightblue LLM Eval Framework: tengu, elyza100, ja-mtbench, rakuda
  ☆18Updated last week
• tsuruoka-lab / BSD
  The Business Scene Dialogue corpus
  ☆71Updated 4 years ago
• amazon-science / amazon-multilingual-counterfactual-dataset
  ☆43Updated 4 years ago
• tsuruoka-lab / AMI-Meeting-Parallel-Corpus
  AMI Meeting Parallel Corpus
  ☆11Updated 4 years ago
• keyreply / Thai-NLP-Dataset
  More than 43+ collections of Thai Natural Language Processing libraries. Update daily.
  ☆30Updated 7 years ago
• amazon-science / contrastive-controlled-mt
  Code and data for the IWSLT 2022 shared task on Formality Control for SLT
  ☆21Updated 2 years ago
• valentinhofmann / flota
  ☆17Updated 2 years ago
• allenai / zest
  Code and dataset "ZEST" from "Learning from task descriptions", Weller et al, EMNLP 2020
  ☆17Updated 4 years ago
• gentaiscool / indonesian-nlp
  A curated list of research papers and resources on Indonesian languages
  ☆39Updated last year
• nlp-waseda / comet-atomic-ja
  COMET-ATOMIC ja
  ☆31Updated last year
• jungokasai / beam_with_patience
  ☆46Updated 3 years ago
• rycolab / artificial-languages
  ☆12Updated 3 years ago
• shijie-wu / neural-transducer
  This repo contains a set of neural transducer, e.g. sequence-to-sequence model, focusing on character-level tasks.
  ☆76Updated 2 years ago
• matbahasa / TALPCo
  TUFS Asian Language Parallel Corpus
  ☆51Updated 2 years ago
• hppRC / defsent
  DefSent: Sentence Embeddings using Definition Sentences
  ☆22Updated 4 years ago
• machelreid / m2d2
  M2D2: A Massively Multi-domain Language Modeling Dataset (EMNLP 2022) by Machel Reid, Victor Zhong, Suchin Gururangan, Luke Zettlemoyer
  ☆54Updated 3 years ago
• fajri91 / sum_liputan6
  The first large-scale summarization corpus for the Indonesian language. AACL 2020.
  ☆38Updated 4 years ago
• gsarti / t5-flax-gcp
  Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP
  ☆58Updated 3 years ago
• MicrosoftTranslator / NTREX
  NTREX -- News Test References for MT Evaluation
  ☆86Updated last year
• ehsanasgari / 1000Langs
  Creating super-parallel corpora of more than 1500+ unique languages for NLP research
  ☆34Updated 2 years ago
• google-research / xtreme-up
  ☆52Updated 2 years ago
• alirezamshi-zz / small100
  Implementation of "SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages" paper, accepted to E…
  ☆25Updated 3 years ago
• Geotrend-research / smaller-transformers
  Load What You Need: Smaller Multilingual Transformers for Pytorch and TensorFlow 2.0.
  ☆105Updated 3 years ago
• esdurmus / Wikilingua
  Multilingual abstractive summarization dataset extracted from WikiHow.
  ☆96Updated 8 months ago
• laboroai / Laboro-ParaCorpus
  Scripts for creating a Japanese-English parallel corpus and training NMT models
  ☆18Updated 4 years ago