microsoft / factored-segmenter
Unsupervised factor-based text tokenizer for natural-language processing applications
☆17Updated 4 years ago
Alternatives and similar repositories for factored-segmenter:
Users that are interested in factored-segmenter are comparing it to the libraries listed below
- Microsoft Speech Language Translation (MSLT) Corpus☆19Updated 7 years ago
- ☆12Updated 9 years ago
- Machine is a natural language processing library for .NET that is focused on providing tools for processing resource-poor languages.☆28Updated this week
- Bicleaner fork that uses neural networks☆39Updated 8 months ago
- C# Word2Vec object with fast neighbor search. Format compatible with gensim☆25Updated 4 years ago
- Port of PragmaticSegmenter for sentence boundary detection☆35Updated 3 years ago
- ☆25Updated last year
- Automatically exported from code.google.com/p/m2m-aligner☆42Updated 8 years ago
- SHAS: Approaching optimal Segmentation for End-to-End Speech Translation☆38Updated 2 years ago
- NTREX -- News Test References for MT Evaluation☆81Updated 9 months ago
- Bilingual sentence similarity classifier using Tensorflow☆21Updated 5 years ago
- SIGMORPHON 2020 Shared Task: Grapheme-to-Phoneme, Unsupervised Induction of Morphology, and Typologically Diverse Morphological Inflectio…☆36Updated 4 years ago
- Tools for filtering and cleaning parallel and monolingual corpora for machine translation and other natural language processing tasks.☆41Updated last year
- Efficient Low-Memory Aligner☆143Updated 2 months ago
- Efficient Markov Chain word alignment☆53Updated 3 years ago
- Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.☆50Updated 3 months ago
- Large scale (>200h) and publicly available read audio book corpus. This corpus is an augmentation of LibriSpeech ASR Corpus (1000h) and c…☆43Updated 2 years ago
- A library for minimum Bayes risk (MBR) decoding☆37Updated last month
- A High-Quality Multilingual Dataset for Structured Documentation Translation☆35Updated 8 months ago
- ☆72Updated this week
- Spoken Language Translation System☆14Updated 5 years ago
- Pure C# port of the Pocketsphinx keyword spotter☆13Updated 5 years ago
- ☆42Updated 6 years ago
- Appraise evaluation system for manual evaluation of machine translation output☆74Updated 3 years ago
- Cog is a tool for comparing languages using lexicostatistics and comparative linguistics techniques.☆23Updated last year
- The Fisher and CALLHOME Spanish–English Speech Translation Corpus☆39Updated 3 years ago
- Covering grammars for English and Russian text normalization☆60Updated 5 years ago
- A bunch of scripts exploiting several tools to perform inverse text normalization (ITN)☆21Updated 7 years ago
- A toolkit for producing n-gram language models. The highlights are the implementation of Kneser-Ney growing and revised Kneser pruning me…☆40Updated 6 months ago
- OpusFilter - Parallel corpus processing toolkit☆104Updated last week