Textprep is an analyzing tool for both parallel and non-parallel corpus and its down-stream Natural Language Processing and Machine Translation tasks. It is designed especially for logographic languages such as Chinese and Japanese.
☆32Feb 25, 2019Updated 7 years ago
Alternatives and similar repositories for textprep
Users that are interested in textprep are comparing it to the libraries listed below
Sorting:
- Deep Inside-Outside Recursive Autoencoder☆89Jan 17, 2022Updated 4 years ago
- We use phonetics as a feature to create a joint semantic-phonetic embedding and improve the neural machine translation between Chinese an…☆12Aug 3, 2021Updated 4 years ago
- Neural Network-based Statistical Machine Translation Toolkit.☆73Jul 24, 2017Updated 8 years ago
- Syntactic evaluation sets, attribute-varying grammars, and code for replicating the CLAMS paper. ACL 2020.☆17Nov 26, 2024Updated last year
- Klexikon: A German Dataset for Joint Summarization and Simplification☆17Oct 5, 2022Updated 3 years ago
- Recursive Neural Tensor Networks☆11Feb 3, 2014Updated 12 years ago
- ☆10Aug 13, 2012Updated 13 years ago
- Tools for sparse networks with the Torch Framework☆12Oct 10, 2016Updated 9 years ago
- Resources for the OpenNMT hackathon☆51May 24, 2019Updated 6 years ago
- ☆20Sep 23, 2018Updated 7 years ago
- benchmarks for LLM tokenizers☆17Feb 27, 2026Updated 3 weeks ago
- A High-Quality Multilingual Dataset for Structured Documentation Translation☆37May 1, 2025Updated 10 months ago
- ☆10Oct 16, 2017Updated 8 years ago
- ☆20Aug 17, 2021Updated 4 years ago
- Zunda: Japanese Enhanced Modality Analyzer client for Python.☆10Nov 30, 2019Updated 6 years ago
- Automatically exported from code.google.com/p/transducersaurus☆11Apr 1, 2015Updated 10 years ago
- FlexiTokens☆18Dec 27, 2025Updated 2 months ago
- AutoML Workshop (Azure Machine Learning mainly)☆13Jan 5, 2020Updated 6 years ago
- C++ code of "Tree-to-Sequence Attentional Neural Machine Translation (tree2seq ANMT)"☆57Jun 23, 2017Updated 8 years ago
- ☆34Nov 29, 2016Updated 9 years ago
- A localized word dictionary asset for University of Tsukuba☆12Sep 19, 2025Updated 6 months ago
- ☆12Jan 21, 2017Updated 9 years ago
- chinese word segmentation based on rnn☆13Oct 14, 2016Updated 9 years ago
- Source code and data for the paper "Towards String-to-Tree Neural Machine Translation"☆16Dec 31, 2017Updated 8 years ago
- full text search engine based on compact data structures☆13Jan 26, 2015Updated 11 years ago
- Code to support the paper "Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets"☆65Aug 31, 2021Updated 4 years ago
- Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation☆15Aug 27, 2024Updated last year
- Cynical data selection☆20Jan 16, 2021Updated 5 years ago
- Multiple Different Natural Language Processing Tasks in a Single Deep Model☆48Dec 5, 2018Updated 7 years ago
- The repository contains the dataset and the code of the paper: Document-Level Text Simplification: Dataset, Metric and Model.☆26Jun 2, 2023Updated 2 years ago
- ☆11Nov 17, 2018Updated 7 years ago
- ☆10Jan 12, 2018Updated 8 years ago
- ☆53Jul 25, 2023Updated 2 years ago
- Implementation of Word Embedding-based Antonym Detection using Thesauri and Distributional Information in NAACL2015☆35Mar 8, 2022Updated 4 years ago
- Nanoloop source files for the album "Prime 16"☆11Mar 7, 2026Updated 2 weeks ago
- Implementation of "Controlling Output Length in Neural Encoder-Decoders"☆42Jan 29, 2018Updated 8 years ago
- ☆12Sep 1, 2021Updated 4 years ago
- auto refresh no-ip free domain, avoid 30 days expiration☆14Aug 1, 2019Updated 6 years ago
- 青空文庫のテキストファイル☆14Feb 4, 2024Updated 2 years ago