VKCOM / YouTokenToMeLinks
Unsupervised text tokenizer focused on computational efficiency
☆967Updated last year
Alternatives and similar repositories for YouTokenToMe
Users that are interested in YouTokenToMe are comparing it to the libraries listed below
Sorting:
- Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE)☆1,211Updated 8 months ago
- Fast BPE☆671Updated 11 months ago
- A python tool for evaluating the quality of sentence embeddings.☆2,108Updated last year
- Fast topic modeling platform☆668Updated last year
- PyTorch original implementation of Cross-lingual Language Model Pretraining.☆2,907Updated 2 years ago
- Web-ify your word2vec: framework to serve distributional semantic models online☆200Updated 3 months ago
- FastFormers - highly efficient transformer models for NLU☆705Updated 2 months ago
- Fast, general, and tested differentiable structured prediction in PyTorch☆1,113Updated 3 years ago
- Super easy library for BERT based NLP models☆1,898Updated 9 months ago
- Plug and Play Language Model implementation. Allows to steer topic and attributes of GPT-2 models.☆1,147Updated last year
- A list of pretrained Transformer models for the Russian language.☆174Updated 5 years ago
- jiant is an nlp toolkit☆1,667Updated last year
- Unsupervised Word Segmentation for Neural Machine Translation and Text Generation☆2,235Updated 9 months ago
- Python port of Moses tokenizer, truecaser and normalizer☆494Updated last year
- Models for automatic abstractive summarization☆171Updated 2 years ago
- A tool for holistic analysis of language generations systems☆468Updated 3 years ago
- MASS: Masked Sequence to Sequence Pre-training for Language Generation☆1,116Updated 2 years ago
- Tools for shrinking fastText models (in gensim format)☆178Updated last year
- Modern spell checking library - accurate, fast, multi-language☆638Updated 9 months ago
- Repository for the paper "Optimal Subarchitecture Extraction for BERT"☆472Updated 2 years ago
- A framework to learn cross-lingual word embedding mappings☆649Updated 2 years ago
- Package for evaluating word embeddings☆436Updated 4 years ago
- A fast, efficient universal vector embedding utility package.☆1,645Updated last year
- An Analysis Toolkit for Natural Language Generation (Translation, Captioning, Summarization, etc.)☆445Updated 2 months ago
- 🌊HMTL: Hierarchical Multi-Task Learning - A State-of-the-Art neural network model for several NLP tasks based on PyTorch and AllenNLP☆1,195Updated last year
- Repository of code for the tutorial on Transfer Learning in NLP held at NAACL 2019 in Minneapolis, MN, USA☆723Updated 5 years ago
- Transformer language model (GPT-2) with sentencepiece tokenizer☆164Updated 4 years ago
- XTREME is a benchmark for the evaluation of the cross-lingual generalization ability of pre-trained multilingual models that covers 40 ty…☆644Updated 2 years ago
- Open-Source Neural Machine Translation in Tensorflow☆799Updated 2 years ago
- Tensorflow implementation of contextualized word representations from bi-directional language models☆1,615Updated 2 years ago