microsoft / CodeMixed-Text-GeneratorLinks
This tool helps automatic generation of grammatically valid synthetic Code-mixed data by utilizing linguistic theories such as Equivalence Constant Theory and Matrix Language Theory.
β55Updated last year
Alternatives and similar repositories for CodeMixed-Text-Generator
Users that are interested in CodeMixed-Text-Generator are comparing it to the libraries listed below
Sorting:
- Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.β83Updated 11 months ago
- Tutorial to pretrain & fine-tune a π€ Flax T5 model on a TPUv3-8 with GCPβ58Updated 3 years ago
- NTREX -- News Test References for MT Evaluationβ85Updated last year
- This code provides word level language identification tool for identifying language for individual words in Code-Mixed text. e.g. The texβ¦β55Updated 5 years ago
- A tiny BERT for low-resource monolingual modelsβ31Updated 11 months ago
- Official Implementation of "DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization."β141Updated 2 years ago
- A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB teβ¦β282Updated 7 months ago
- Reduce the size of pretrained Hugging Face models via vocabulary trimming.β45Updated 2 years ago
- Code and models used in "MUSS Multilingual Unsupervised Sentence Simplification by Mining Paraphrases".β99Updated 2 years ago
- Code and data for the IWSLT 2022 shared task on Formality Control for SLTβ21Updated 2 years ago
- BLOOM+1: Adapting BLOOM model to support a new unseen languageβ73Updated last year
- [EMNLP 2021] LM-Critic: Language Models for Unsupervised Grammatical Error Correctionβ120Updated 3 years ago
- This repo contains a set of neural transducer, e.g. sequence-to-sequence model, focusing on character-level tasks.β76Updated last year
- Yet Another Neural Machine Translation Toolkitβ179Updated 5 months ago
- A flexible sentence segmentation library using CRF model and regex rulesβ29Updated last year
- Complimentary code for our paper Automatic punctuation restoration with BERT modelsβ50Updated last year
- Bicleaner fork that uses neural networksβ40Updated 2 months ago
- FactSumm: Factual Consistency Scorer for Abstractive Summarizationβ112Updated last year
- Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023β103Updated last year
- Pytorch Implementation of EncT5: Fine-tuning T5 Encoder for Non-autoregressive Tasksβ63Updated 3 years ago
- OpusFilter - Parallel corpus processing toolkitβ110Updated 3 weeks ago
- Repository containing the open source code of works published at the FBK MT unit.β48Updated last month
- Build a dialog dataset from online books in many languagesβ76Updated 2 years ago
- β103Updated 4 years ago
- cLang-8 is a dataset for grammatical error correction.β107Updated 3 years ago
- Tools for evaluating the performance of MT metrics on data from recent WMT metrics shared tasks.β114Updated 5 months ago
- Megatron LM 11B on Huggingface Transformersβ27Updated 4 years ago
- Repository for SLURP paperβ105Updated 3 years ago
- [TMLR'23] Contrastive Search Is What You Need For Neural Text Generationβ119Updated 2 years ago
- The FLORES+ Machine Translation Benchmarkβ108Updated 9 months ago