modernmt / DataCollection
Data collection, alignment and TAUS repository
☆23Updated 7 years ago
Alternatives and similar repositories for DataCollection:
Users that are interested in DataCollection are comparing it to the libraries listed below
- ☆42Updated 6 years ago
- ☆12Updated 9 years ago
- BERT models for many languages created from Wikipedia texts☆33Updated 4 years ago
- Tool for comparison and evaluation of machine translation.☆56Updated 2 years ago
- Tools for extracting parallel corpora from article titles across languages in Wikipedia☆72Updated 10 years ago
- ☆23Updated 7 years ago
- Translation Memory Open-source Purifier☆33Updated 2 years ago
- Appraise evaluation system for manual evaluation of machine translation output☆74Updated 3 years ago
- DKPro C4CorpusTools is a collection of tools for processing CommonCrawl corpus, including Creative Commons license detection, boilerplate…☆51Updated 4 years ago
- ☆21Updated 5 years ago
- Crawling engine that crawls a set of top-level domains looking for documents in a list of languages☆10Updated last year
- Thot toolkit for statistical machine translation☆53Updated 2 years ago
- Tool to fix bitexts and tag near-duplicates for removal☆30Updated last month
- Code for the paper Faster Phrase-Based Decoding by Refining Feature State☆14Updated 2 years ago
- A simple neural truecaser written in pytorch and allennlp.☆33Updated 8 months ago
- A library for data streaming and augmentation☆20Updated 11 months ago
- An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…☆22Updated 3 years ago
- Transform TMX to text☆28Updated 2 years ago
- Neural macine translation soft alignment visualisations for web and command line☆72Updated 3 years ago
- Bilingual sengence aligner☆27Updated last year
- A parallel evaluation data set of SAP software documentation with document structure annotation☆10Updated 6 months ago
- Bilingual term extractor☆53Updated last year
- numeric fused-head identification and resolution☆33Updated 5 years ago
- Multilingual NLP annotation projection☆51Updated 2 years ago
- Efficient Markov Chain word alignment☆53Updated 3 years ago
- Program used to split text into segments☆25Updated 4 months ago
- Open-source tools for morphological tagging, segmentation and stemming.☆41Updated 5 years ago
- These are lists for a variety of languages containing words that are distinctive to each language.☆36Updated 2 years ago
- Decoding platform for machine translation research☆54Updated 5 years ago
- Efficient Low-Memory Aligner☆142Updated last month