levyfan / sentencepiece-jni
Java JNI wrapper for SentencePiece: unsupervised text tokenizer for Neural Network-based text generation.
☆37Updated 2 years ago
Alternatives and similar repositories for sentencepiece-jni
Users that are interested in sentencepiece-jni are comparing it to the libraries listed below
Sorting:
- Subword Language Model for Query Auto-Completion☆67Updated 5 years ago
- ☆21Updated 5 years ago
- Tools for training pytorch language models☆27Updated 4 years ago
- [EMNLP 2021] LM-Critic: Language Models for Unsupervised Grammatical Error Correction☆119Updated 3 years ago
- Data and code for the paper "End-to-End Slot Alignment and Recognition for Cross-Lingual NLU" (Accepted to EMNLP 2020)☆25Updated 3 years ago
- Implementation of pQRNN in PyTorch☆46Updated 3 years ago
- Decoding platform for machine translation research☆55Updated 5 years ago
- reference pytorch code for intent classification☆44Updated 7 months ago
- Neural network models for joint POS tagging and dependency parsing (CoNLL 2017-2018)☆158Updated 5 years ago
- Code and datasets of "Multilingual Extractive Reading Comprehension by Runtime Machine Translation"☆40Updated 6 years ago
- Java port of c++ version of facebook fasttext☆14Updated 5 years ago
- An implementation of BERT using PyTorch's TransformerEncoder☆33Updated 5 years ago
- Phrase-Indexed Question Answering (PIQA)☆94Updated 6 years ago
- LM Pretraining with PyTorch/TPU☆134Updated 5 years ago
- ☆42Updated 6 years ago
- BERT models for many languages created from Wikipedia texts☆33Updated 4 years ago
- Team Kakao&Brain's Grammatical Error Correction System for the ACL 2019 BEA Shared Task☆92Updated 5 years ago
- Automatic extraction of edited sentences from text edition histories.☆83Updated 3 years ago
- A parser of the Multi-Domain Wizard-of-Oz dataset (MultiWOZ)☆67Updated 6 years ago
- MaxMatch (M^2) Scorer - Evaluation program for grammatical error correction systems.☆151Updated 2 years ago
- A collection of resources on using BERT (https://arxiv.org/abs/1810.04805 ) and related Language Models in production environments.☆96Updated 4 years ago
- On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines☆136Updated last year
- Word Piece Model python light version with functions tokenize/save/load☆65Updated 4 years ago
- Supplementary material for "When and Why Are Pre-trained Word Embeddings Useful for Neural Machine Translation?" at NAACL 2018☆122Updated 5 years ago
- We release a dataset based on Wikipedia sentences and the corresponding translations in 6 different languages along with the scores (scal…☆81Updated 3 years ago
- ☆32Updated 3 years ago
- A statistical machine translation (SMT)-based grammatical error correction system that makes use of neural network joint models (NNJM) an…☆25Updated 6 years ago
- This repo includes extensions to the Stanford Dialogue Corpus. It contains crowd-sourced rewrites to facilitate research in dialogue stat…☆90Updated 5 years ago
- This is kinda convoluted re-implementation of ELECTRA☆24Updated 5 years ago
- Assessing syntactic abilities of BERT☆39Updated 5 years ago