VKCOM / YouTokenToMe
Unsupervised text tokenizer focused on computational efficiency
☆965Updated 11 months ago
Alternatives and similar repositories for YouTokenToMe:
Users that are interested in YouTokenToMe are comparing it to the libraries listed below
- Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE)☆1,204Updated 5 months ago
- Fast BPE☆666Updated 8 months ago
- A tool for holistic analysis of language generations systems☆467Updated 2 years ago
- Modern spell checking library - accurate, fast, multi-language☆630Updated 6 months ago
- Fast, general, and tested differentiable structured prediction in PyTorch☆1,111Updated 2 years ago
- FastFormers - highly efficient transformer models for NLU☆704Updated last year
- Plug and Play Language Model implementation. Allows to steer topic and attributes of GPT-2 models.☆1,141Updated last year
- Language-Agnostic SEntence Representations☆3,619Updated 10 months ago
- PyTorch original implementation of Cross-lingual Language Model Pretraining.☆2,900Updated 2 years ago
- Super easy library for BERT based NLP models☆1,886Updated 6 months ago
- Python port of Moses tokenizer, truecaser and normalizer☆490Updated 9 months ago
- Open-Source Neural Machine Translation in Tensorflow☆797Updated 2 years ago
- A python tool for evaluating the quality of sentence embeddings.☆2,096Updated 11 months ago
- 🌊HMTL: Hierarchical Multi-Task Learning - A State-of-the-Art neural network model for several NLP tasks based on PyTorch and AllenNLP☆1,193Updated last year
- jiant is an nlp toolkit☆1,666Updated last year
- MASS: Masked Sequence to Sequence Pre-training for Language Generation☆1,118Updated 2 years ago
- A fast, efficient universal vector embedding utility package.☆1,643Updated last year
- Tools for shrinking fastText models (in gensim format)☆178Updated 10 months ago
- A list of pretrained Transformer models for the Russian language.☆173Updated 5 years ago
- Repository of code for the tutorial on Transfer Learning in NLP held at NAACL 2019 in Minneapolis, MN, USA☆722Updated 5 years ago
- Models for automatic abstractive summarization☆171Updated 2 years ago
- LASER multilingual sentence embeddings as a pip package☆224Updated last year
- Beyond Accuracy: Behavioral Testing of NLP models with CheckList☆2,029Updated last year
- Single Headed Attention RNN - "Stop thinking with your head"☆1,181Updated 3 years ago
- Web-ify your word2vec: framework to serve distributional semantic models online☆199Updated 3 weeks ago
- Obtain Word Alignments using Pretrained Language Models (e.g., mBERT)☆359Updated last year
- 💥 Use the latest Stanza (StanfordNLP) research models directly in spaCy☆731Updated 6 months ago
- Deep NLP Course☆630Updated 5 years ago
- An efficient implementation of the popular sequence models for text generation, summarization, and translation tasks. https://arxiv.org/p…☆430Updated 2 years ago
- NL-Augmenter 🦎 → 🐍 A Collaborative Repository of Natural Language Transformations☆780Updated 9 months ago