Alternatives and similar repositories for seed:

Users that are interested in seed are comparing it to the libraries listed below

• openlanguagedata / flores
  The FLORES+ Machine Translation Benchmark
  ☆100Updated 3 months ago
• alirezamshi / small100
  Implementation of "SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages" paper, accepted to E…
  ☆25Updated 2 years ago
• alirezamshi-zz / small100
  Implementation of "SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages" paper, accepted to E…
  ☆21Updated 2 years ago
• MicrosoftTranslator / NTREX
  NTREX -- News Test References for MT Evaluation
  ☆81Updated 8 months ago
• laurieburchell / open-lid-dataset
  Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)
  ☆70Updated 10 months ago
• MeLeLBGU / SaGe
  Code for SaGe subword tokenizer (EACL 2023)
  ☆24Updated 3 months ago
• bigscience-workshop / multilingual-modeling
  BLOOM+1: Adapting BLOOM model to support a new unseen language
  ☆70Updated last year
• kpu / fasterText
  Library for fast text representation and classification.
  ☆28Updated last year
• cisnlp / Glot500
  Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023
  ☆99Updated 10 months ago
• steinst / SentAlign
  ☆25Updated last year
• codogogo / towerparse
  Tower Parse: Low-Resource Dependency Parsing via Hierarchical Source Selection
  ☆15Updated 3 years ago
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated last year
• SYSTRAN / fuzzy-match
  Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.
  ☆47Updated 2 months ago
• wikimedia / sentencex
  A sentence segmentation library with wide language support optimized for speed and utility.
  ☆58Updated 6 months ago
• google-research / metricx
  ☆83Updated 2 months ago
• epfml / Bi-Sent2Vec
  Robust Cross-lingual Embeddings from Parallel Sentences
  ☆22Updated 4 years ago
• bitextor / warc2text
  Extracts plain text, language identification and more metadata from WARC records
  ☆21Updated 3 weeks ago
• lgessler / microbert
  A tiny BERT for low-resource monolingual models
  ☆31Updated 5 months ago
• hplt-project / OpusCleaner
  OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.
  ☆49Updated last month
• shyyhs / CourseraParallelCorpusMining
  Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation
  ☆14Updated 6 months ago
• gautierdag / tokenizer-bench
  Code for the paper "Getting the most out of your tokenizer for pre-training and domain adaptation"
  ☆16Updated last year
• cisnlp / GlotLID
  Language Identification with Support for More Than 2000 Labels -- EMNLP 2023
  ☆118Updated 3 months ago
• ltgoslo / ltg-bert
  LTG-Bert
  ☆29Updated last year
• applicaai / CCpdf
  Index of URLs to pdf files all over the internet and scripts
  ☆21Updated last year
• Helsinki-NLP / OPUS
  The Open Parallel Corpus
  ☆64Updated this week
• Helsinki-NLP / OpusFilter
  OpusFilter - Parallel corpus processing toolkit
  ☆104Updated this week
• malteos / llm-datasets
  A collection of datasets for language model pretraining including scripts for downloading, preprocesssing, and sampling.
  ☆56Updated 7 months ago
• machinetranslate / machinetranslate.org
  Open information and community for machine translation
  ☆74Updated 2 weeks ago
• marian-nmt / sotastream
  A library for data streaming and augmentation
  ☆20Updated 11 months ago
• guilhermemr04 / scaling-zero-shot-retrieval
  No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval
  ☆28Updated 2 years ago