Alternatives and similar repositories for Opensubtitles_dataset:

Users that are interested in Opensubtitles_dataset are comparing it to the libraries listed below

• bitextor / bifixer
  Tool to fix bitexts and tag near-duplicates for removal
  ☆29Updated 2 weeks ago
• hplt-project / OpusCleaner
  OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.
  ☆49Updated last month
• bitextor / bicleaner-ai
  Bicleaner fork that uses neural networks
  ☆39Updated 6 months ago
• lucidrains / charformer-pytorch
  Implementation of the GBST block from the Charformer paper, in Pytorch
  ☆117Updated 3 years ago
• hplt-project / OpusTrainer
  Curriculum training
  ☆16Updated last month
• EleutherAI / openwebtext2
  ☆89Updated 2 years ago
• neulab / newlang-tech
  A guide to building language technology in new languages.
  ☆58Updated 3 years ago
• CPJKU / wechsel
  Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.
  ☆77Updated 5 months ago
• lgessler / microbert
  A tiny BERT for low-resource monolingual models
  ☆31Updated 4 months ago
• Helsinki-NLP / OpusFilter
  OpusFilter - Parallel corpus processing toolkit
  ☆104Updated 3 weeks ago
• thammegowda / mtdata
  A tool that locates, downloads, and extracts machine translation corpora
  ☆150Updated 8 months ago
• laurieburchell / open-lid-dataset
  Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)
  ☆70Updated 9 months ago
• daltonfury42 / truecase
  A python true casing utility that restores case information for texts
  ☆88Updated 2 years ago
• oscar-project / goclassy
  An asynchronous concurrent pipeline for classifying Common Crawl based on fastText's pipeline.
  ☆86Updated 3 years ago
• gsarti / t5-flax-gcp
  Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP
  ☆58Updated 2 years ago
• microsoft / xtreme-distil-transformers
  XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale
  ☆154Updated last year
• bitextor / bicleaner
  Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.
  ☆154Updated 8 months ago
• Felflare / rpunct
  📝An easy-to-use package to restore punctuation of the text.
  ☆112Updated last year
• oscar-project / ungoliant
  The pipeline for the OSCAR corpus
  ☆166Updated last year
• Helsinki-NLP / subalign
  ☆15Updated last year
• deep-spin / tower-eval
  ☆28Updated this week
• Helsinki-NLP / OpusTools
  ☆71Updated 2 weeks ago
• google-research-datasets / TF-IDF-IIF-top100-wordlists
  These are lists for a variety of languages containing words that are distinctive to each language.
  ☆35Updated 2 years ago
• wangcongcong123 / ttt
  A package for fine-tuning Transformers with TPUs, written in Tensorflow2.0+
  ☆37Updated 3 years ago
• huggingface / olm-datasets
  Pipeline for pulling and processing online language model pretraining data from the web
  ☆175Updated last year
• facebookresearch / muss
  Code and models used in "MUSS Multilingual Unsupervised Sentence Simplification by Mining Paraphrases".
  ☆98Updated 2 years ago
• rewicks / ersatz
  ☆45Updated 6 months ago
• snukky / wikiedits
  Automatic extraction of edited sentences from text edition histories.
  ☆82Updated 3 years ago
• MicrosoftTranslator / NTREX
  NTREX -- News Test References for MT Evaluation
  ☆81Updated 8 months ago
• bitextor / warc2text
  Extracts plain text, language identification and more metadata from WARC records
  ☆21Updated 2 weeks ago