microsoft / factored-segmenter
Unsupervised factor-based text tokenizer for natural-language processing applications
☆17Updated 4 years ago
Related projects ⓘ
Alternatives and complementary repositories for factored-segmenter
- Microsoft Speech Language Translation (MSLT) Corpus☆19Updated 7 years ago
- ☆31Updated 2 years ago
- Machine is a natural language processing library for .NET that is focused on providing tools for processing resource-poor languages.☆27Updated this week
- Port of PragmaticSegmenter for sentence boundary detection☆33Updated 3 years ago
- Bicleaner fork that uses neural networks☆38Updated 3 months ago
- Pure C# port of the Pocketsphinx keyword spotter☆12Updated 4 years ago
- ☆91Updated 9 months ago
- ☆22Updated last year
- A library for minimum Bayes risk (MBR) decoding☆31Updated last week
- A High-Quality Multilingual Dataset for Structured Documentation Translation☆34Updated 4 months ago
- C# Word2Vec object with fast neighbor search. Format compatible with gensim☆25Updated 4 years ago
- Large scale (>200h) and publicly available read audio book corpus. This corpus is an augmentation of LibriSpeech ASR Corpus (1000h) and c…☆43Updated 2 years ago
- NTREX -- News Test References for MT Evaluation☆75Updated 5 months ago
- An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…☆21Updated 2 years ago
- A library for data streaming and augmentation☆20Updated 8 months ago
- .NET researching application for rendering, recording, playback, analyzing and compression of audio data.☆10Updated 7 years ago
- SIGMORPHON 2020 Shared Task: Grapheme-to-Phoneme, Unsupervised Induction of Morphology, and Typologically Diverse Morphological Inflectio…☆35Updated 3 years ago
- This repo contains a set of neural transducer, e.g. sequence-to-sequence model, focusing on character-level tasks.☆72Updated last year
- This code provides word level language identification tool for identifying language for individual words in Code-Mixed text. e.g. The tex…☆50Updated 4 years ago
- An adaptation of Fairseq to (End-to-end) speech translation.☆22Updated 2 years ago
- SHAS: Approaching optimal Segmentation for End-to-End Speech Translation☆37Updated last year
- Tools for filtering and cleaning parallel and monolingual corpora for machine translation and other natural language processing tasks.☆41Updated 11 months ago
- Labeled data for homograph disambiguation☆53Updated last year
- ☆67Updated 3 months ago
- A phonetic matching library. Includes text utilities to do string comparisons on phonemes (the sound of the string), as opposed to charac…☆157Updated last year
- ☆14Updated 4 years ago
- c++ mosestokenizer☆16Updated 8 months ago
- Efficient Low-Memory Aligner☆139Updated 2 months ago
- SIGMORPHON 2022 Shared Task on Morpheme Segmentation☆24Updated last year
- Japanese--Russian--English News Commentary Parallel Data☆18Updated 5 years ago