Byte Pair Encoding for Python!
☆232Sep 16, 2022Updated 3 years ago
Alternatives and similar repositories for python-bpe
Users that are interested in python-bpe are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Unsupervised Word Segmentation for Neural Machine Translation and Text Generation☆2,267Aug 7, 2024Updated last year
- Calculating Expected Time for training LLM.☆39Apr 17, 2023Updated 2 years ago
- ☆16Sep 14, 2019Updated 6 years ago
- Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE)☆1,221Oct 1, 2024Updated last year
- smoothed box embedding code☆16Apr 5, 2020Updated 6 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- KoParadigm: Korean Inflectional Paradigm Generator☆57Nov 23, 2022Updated 3 years ago
- Fast BPE☆678Jun 18, 2024Updated last year
- some tutorials for blog: simonjisu.github.io☆23Mar 25, 2021Updated 5 years ago
- (re)Implementation of Learning Multi-level Dependencies for Robust Word Recognition☆17Jul 25, 2024Updated last year
- exBERT on Transformers🤗☆10Jun 14, 2021Updated 4 years ago
- Memory-efficient Count-Min Sketch Counter (based on Madoka C++ library)☆27Nov 30, 2025Updated 4 months ago
- Code for Context is Everything: Finding Meaning Statistically in Semantic Spaces.☆16Apr 28, 2018Updated 7 years ago
- Jasper 기반 양자화된 모델인 Quartznet 한국어 음성인식☆22Jul 21, 2021Updated 4 years ago
- Roundtrip translation (aka back translation) python package☆14Dec 8, 2022Updated 3 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- A repository comprising of code for generation of noisy speech data from clean data using deep learning methods☆16Jul 12, 2021Updated 4 years ago
- Code for the paper "Language Models are Unsupervised Multitask Learners"☆20Jan 7, 2020Updated 6 years ago
- ☆16Nov 26, 2024Updated last year
- Scripts investigate, repair and normalize a wide range of text file problems at the character level.☆23Nov 23, 2022Updated 3 years ago
- Pre-trained Machine Translation Models of Korean from/to ECJ☆29Jul 15, 2019Updated 6 years ago
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators☆2,373Mar 23, 2024Updated 2 years ago
- Fast and customizable tokenization☆67Jul 9, 2019Updated 6 years ago
- ☆10Aug 25, 2020Updated 5 years ago
- Keras library for building (Universal) Transformers, facilitating BERT and GPT models☆541May 30, 2020Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Anh - LAION's multilingual assistant datasets and models☆28Apr 5, 2023Updated 3 years ago
- Unsupervised parallel sentence extraction from comparable corpora☆16Aug 6, 2019Updated 6 years ago
- Contains the code and data for our #ICSE2022 paper titled as "CodeFill: Multi-token Code Completion by Jointly Learning from Structure an…☆15May 18, 2022Updated 3 years ago
- A lightweight but powerful library to build token indices for NLP tasks, compatible with major Deep Learning frameworks like PyTorch and …☆50Dec 6, 2024Updated last year
- 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.☆81Mar 17, 2022Updated 4 years ago
- Introduction Notebook to Extreme Multi-Label Classification problem (XML)☆22Sep 9, 2018Updated 7 years ago
- Korean text normalization and language preparation package for LM in Kaldi-based ASR system☆63Apr 23, 2020Updated 5 years ago
- Refactor your software using programming language independent, case-preserving string replacement☆17Jul 9, 2019Updated 6 years ago
- Data and code for Kang et al., EMNLP 2019's paper titled "(Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Ann…☆30Mar 17, 2020Updated 6 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- An unofficial PyTorch implementation of Mix-Phoneme-Bert☆40Jul 10, 2023Updated 2 years ago
- Longformer: The Long-Document Transformer☆2,190Feb 8, 2023Updated 3 years ago
- Word Piece Model python light version with functions tokenize/save/load☆64Oct 1, 2020Updated 5 years ago
- (semi) Grapheme-to-Phoneme (G2P) - seq2seq model using PyTorch for Korean☆23Dec 17, 2017Updated 8 years ago
- Dependency Parsing as Sequence Labeling☆27Jul 25, 2024Updated last year
- Scripts to train a bidirectional LSTM with knowledge distillation from BERT☆159Nov 21, 2019Updated 6 years ago
- Experiments with self-supervised learning☆11Mar 9, 2020Updated 6 years ago