soaxelbrooke / python-bpe
Byte Pair Encoding for Python!
☆227Updated 2 years ago
Alternatives and similar repositories for python-bpe:
Users that are interested in python-bpe are comparing it to the libraries listed below
- Fast BPE☆662Updated 7 months ago
- Neural Text Generation with Unlikelihood Training☆309Updated 3 years ago
- ☆319Updated 2 years ago
- A tool for holistic analysis of language generations systems☆467Updated 2 years ago
- Easily fine tune GPT-2 to fill in missing text☆196Updated 2 years ago
- Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)☆326Updated last year
- Neat (Neural Attention) Vision, is a visualization tool for the attention mechanisms of deep-learning models for Natural Language Process…☆250Updated 6 years ago
- Scripts to train a bidirectional LSTM with knowledge distillation from BERT☆158Updated 5 years ago
- Easy to use NLP library built on PyTorch and TorchText☆254Updated 5 years ago
- LM Pretraining with PyTorch/TPU☆134Updated 5 years ago
- Python library & examples for Masked Language Model Scoring (ACL 2020)☆340Updated 2 years ago
- Python port of Moses tokenizer, truecaser and normalizer☆490Updated 8 months ago
- A toolkit for evaluating the linguistic knowledge and transferability of contextual representations. Code for "Linguistic Knowledge and T…☆210Updated 3 years ago
- A masked language modeling objective to train a model to predict any subset of the target words, conditioned on both the input text and a…☆242Updated 3 years ago
- Large corpus of uncompressed and compressed sentences from news articles.☆123Updated 7 years ago
- Unsupervised Statistical Machine Translation☆229Updated 4 years ago
- XLNet for generating language.☆165Updated 4 years ago
- SentAugment is a data augmentation technique for NLP that retrieves similar sentences from a large bank of sentences. It can be used in c…☆361Updated 2 years ago
- PyTorch Implementation of "Non-Autoregressive Neural Machine Translation"☆268Updated 3 years ago
- Fork of huggingface/pytorch-pretrained-BERT for BERT on STILTs☆106Updated 2 years ago
- Transformers without Tears: Improving the Normalization of Self-Attention☆130Updated 8 months ago
- Create interactive textual heat maps for Jupiter notebooks☆196Updated 8 months ago
- Python code for various NLP metrics☆166Updated 5 years ago
- Full Python implementation of the ROUGE metric, producing same results as in the official perl implementation.☆157Updated 5 years ago
- Variational Methods for Pretraining in Resource-limited Environments☆174Updated 4 years ago
- Concatenated Power Mean Embeddings as Universal Cross-Lingual Sentence Representations☆185Updated 4 years ago
- A Python wrapper for the ROUGE summarization evaluation package☆251Updated 4 years ago
- A Corpus for Multilingual Document Classification in Eight Languages.☆151Updated 2 years ago
- A framework to learn cross-lingual word embedding mappings☆647Updated last year
- Builds wordpiece(subword) vocabulary compatible for Google Research's BERT☆227Updated 4 years ago