microsoft / factored-segmenter
Unsupervised factor-based text tokenizer for natural-language processing applications
☆17Updated 4 years ago
Alternatives and similar repositories for factored-segmenter:
Users that are interested in factored-segmenter are comparing it to the libraries listed below
- A library for minimum Bayes risk (MBR) decoding☆35Updated 3 weeks ago
- Port of PragmaticSegmenter for sentence boundary detection☆34Updated 3 years ago
- Bicleaner fork that uses neural networks☆39Updated 6 months ago
- MorphyNet: a Large Multilingual Database of Derivational and Inflectional Morphology (+morpheme segmentation)☆40Updated last year
- ☆25Updated last year
- Microsoft Speech Language Translation (MSLT) Corpus☆19Updated 7 years ago
- NTREX -- News Test References for MT Evaluation☆81Updated 8 months ago
- SHAS: Approaching optimal Segmentation for End-to-End Speech Translation☆38Updated 2 years ago
- Machine is a natural language processing library for .NET that is focused on providing tools for processing resource-poor languages.☆28Updated this week
- A High-Quality Multilingual Dataset for Structured Documentation Translation☆35Updated 7 months ago
- OPUS-CAT is a collection of software which make it possible to OPUS-MT neural machine translation models in professional translation. OPU…☆75Updated 2 weeks ago
- Large scale (>200h) and publicly available read audio book corpus. This corpus is an augmentation of LibriSpeech ASR Corpus (1000h) and c…☆43Updated 2 years ago
- ☆71Updated 2 weeks ago
- ☆12Updated 9 years ago
- Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.☆47Updated last month
- XenC: open-source data selection tool for NLP☆60Updated 8 years ago
- Bilingual sentence similarity classifier using Tensorflow☆20Updated 5 years ago
- SIGMORPHON 2022 Shared Task on Morpheme Segmentation☆24Updated last year
- An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…☆22Updated 3 years ago
- SIGMORPHON 2020 Shared Task: Grapheme-to-Phoneme, Unsupervised Induction of Morphology, and Typologically Diverse Morphological Inflectio…☆35Updated 4 years ago
- Repository for sharing the data in the Tamasheq language, one of the target languages for the low-resource speech translation track at IW…☆17Updated 2 years ago
- Tools for filtering and cleaning parallel and monolingual corpora for machine translation and other natural language processing tasks.☆41Updated last year
- An adaptation of Fairseq to (End-to-end) speech translation.☆22Updated 2 years ago
- Efficient Low-Memory Aligner☆141Updated last month
- Multilingual sentence alignment using sentence embeddings☆108Updated 3 months ago
- Numpy style scientific programming package☆18Updated 5 years ago
- OpusFilter - Parallel corpus processing toolkit☆104Updated 3 weeks ago
- phone inventory library☆16Updated last year
- ☆42Updated 6 years ago
- Curriculum training☆16Updated last month