jermp / tongrams_estimationLinks
A C++ library implementing fast language models estimation using the 1-Sort algorithm.
☆17Updated 2 years ago
Alternatives and similar repositories for tongrams_estimation
Users that are interested in tongrams_estimation are comparing it to the libraries listed below
Sorting:
- A C++ library providing fast language model queries in compressed space.☆132Updated 2 years ago
- Utilities for manipulating finite state transducers with the OpenFst library.☆32Updated 8 years ago
- finite-state toolkit, EM and Bayesian (Gibbs sampling) training for FST and context-free derivation forests☆41Updated 3 years ago
- zero-vocab or low-vocab embeddings☆18Updated 3 years ago
- Efficient and effective query auto-completion in C++.☆57Updated 2 years ago
- Compute the most likely permutation of a lattice given an LM☆10Updated 12 years ago
- Fast stand-alone C++ decoder for RNN-based NMT models☆30Updated 5 years ago
- Fast and versatile tokenizer for language models, compatible with SentencePiece, Tokenizers, Tiktoken and more. Supports BPE, Unigram and…☆39Updated 2 months ago
- A Translation Task using TurboTransformers☆11Updated 5 years ago
- The repository for the paper "When Do You Need Billions of Words of Pretraining Data?"☆21Updated 5 years ago
- Supporting example for "A Rust SentencePiece implementation"☆20Updated 5 years ago
- Fast SymSpell written in c++ and exposes to python via pybind11☆44Updated 9 months ago
- Deep learning model of machine translation using attentional and structural biases☆13Updated 8 years ago
- Attentional Neural Network that translates text to phones.☆11Updated 7 years ago
- MozoLM: A language model (LM) serving library☆47Updated this week
- Converter from UD-trees to BART representation☆36Updated last year
- A full-text error corrector for English based on transformers and deep learning☆10Updated 2 years ago
- This repo contains code to reproduce some of the results presented in the paper "SentenceMIM: A Latent Variable Language Model"☆28Updated 3 years ago
- Self-contained Python package for OpenFst☆51Updated 2 years ago
- Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.☆50Updated 8 months ago
- CS224S Course Project☆14Updated 11 years ago
- A database of number names for 186 languages, locales, and scripts☆67Updated 2 years ago
- PyTorch speech2text inference script for the NVidia openseq2seq wav2letter model variant☆10Updated 6 years ago
- UniParse: A universal graph-based parsing toolkit☆10Updated 6 years ago
- BERT models for many languages created from Wikipedia texts☆33Updated 5 years ago
- A Python package to facilitate research on building and evaluating automated scoring models.☆71Updated 11 months ago
- An Efficient Language Model Using Double-Array Structures☆17Updated 5 years ago
- Language Model Fine-tuning for Moby Dick☆42Updated 6 years ago
- Fine-Tuning Pre-trained Transformers into Decaying Fast Weights☆19Updated 3 years ago
- Unicode Standard tokenization routines and orthography profile segmentation☆38Updated 10 months ago