Byte Pair Encoding for Python!
☆234Sep 16, 2022Updated 3 years ago
Alternatives and similar repositories for python-bpe
Users that are interested in python-bpe are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Unsupervised Word Segmentation for Neural Machine Translation and Text Generation☆2,271Aug 7, 2024Updated last year
- Calculating Expected Time for training LLM.☆39Apr 17, 2023Updated 3 years ago
- ☆16Sep 14, 2019Updated 6 years ago
- Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE)☆1,222Oct 1, 2024Updated last year
- ☆23Oct 17, 2024Updated last year
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Source code for the paper "Morphological Inflection Generation with Hard Monotonic Attention"☆38Jun 9, 2018Updated 8 years ago
- some tutorials for blog: simonjisu.github.io☆23Mar 25, 2021Updated 5 years ago
- KoParadigm: Korean Inflectional Paradigm Generator☆60Nov 23, 2022Updated 3 years ago
- smoothed box embedding code☆16Apr 5, 2020Updated 6 years ago
- Fast BPE☆677Jun 18, 2024Updated 2 years ago
- (re)Implementation of Learning Multi-level Dependencies for Robust Word Recognition☆17Jul 25, 2024Updated 2 years ago
- exBERT on Transformers🤗☆10Jun 14, 2021Updated 5 years ago
- Code for Context is Everything: Finding Meaning Statistically in Semantic Spaces.☆16Apr 28, 2018Updated 8 years ago
- Jasper 기반 양자화된 모델인 Quartznet 한국어 음성인식☆22Jul 21, 2021Updated 5 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Roundtrip translation (aka back translation) python package☆14Dec 8, 2022Updated 3 years ago
- Memory-efficient Count-Min Sketch Counter (based on Madoka C++ library)☆27Nov 30, 2025Updated 7 months ago
- A repository comprising of code for generation of noisy speech data from clean data using deep learning methods☆16Jul 12, 2021Updated 5 years ago
- Scripts investigate, repair and normalize a wide range of text file problems at the character level.☆23May 25, 2026Updated last month
- Code for the paper "Language Models are Unsupervised Multitask Learners"☆20Jan 7, 2020Updated 6 years ago
- Token and Sentence Level Classification with Google's BERT (TensorFlow)☆10Jul 11, 2019Updated 7 years ago
- Pre-trained Machine Translation Models of Korean from/to ECJ☆28Jul 15, 2019Updated 7 years ago
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators☆2,367Mar 23, 2024Updated 2 years ago
- ☆16Nov 26, 2024Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Unsupervised text tokenizer for Neural Network-based text generation.☆11,978Updated this week
- Unsupervised parallel sentence extraction from comparable corpora☆16Aug 6, 2019Updated 6 years ago
- Keras library for building (Universal) Transformers, facilitating BERT and GPT models☆540May 30, 2020Updated 6 years ago
- ☆10Aug 25, 2020Updated 5 years ago
- Anh - LAION's multilingual assistant datasets and models☆28Apr 5, 2023Updated 3 years ago
- A lightweight but powerful library to build token indices for NLP tasks, compatible with major Deep Learning frameworks like PyTorch and …☆50Dec 6, 2024Updated last year
- Korean text normalization and language preparation package for LM in Kaldi-based ASR system☆64Apr 23, 2020Updated 6 years ago
- Introduction Notebook to Extreme Multi-Label Classification problem (XML)☆22Sep 9, 2018Updated 7 years ago
- 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.☆81Mar 17, 2022Updated 4 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- working on parallel wavenet☆25Apr 19, 2018Updated 8 years ago
- Data and code for Kang et al., EMNLP 2019's paper titled "(Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Ann…☆30Mar 17, 2020Updated 6 years ago
- Longformer: The Long-Document Transformer☆2,201Feb 8, 2023Updated 3 years ago
- Jejueo Datasets for Machine Translation and Speech Synthesis☆83Feb 19, 2020Updated 6 years ago
- Word Piece Model python light version with functions tokenize/save/load☆64Oct 1, 2020Updated 5 years ago
- Scripts to train a bidirectional LSTM with knowledge distillation from BERT☆159Nov 21, 2019Updated 6 years ago
- A pure-python implementation of BK-Trees☆17Feb 19, 2023Updated 3 years ago