Alternatives and similar repositories for kenlm

Users that are interested in kenlm are comparing it to the libraries listed below

• bheinzerling / bpemb
  Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE)
  ☆1,217Updated last year
• rsennrich / subword-nmt
  Unsupervised Word Segmentation for Neural Machine Translation and Text Generation
  ☆2,256Updated last year
• moses-smt / mosesdecoder
  Moses, the machine translation system
  ☆1,617Updated 7 months ago
• facebookresearch / SentEval
  A python tool for evaluating the quality of sentence embeddings.
  ☆2,105Updated last year
• OpenNMT / OpenNMT-tf
  Neural machine translation and sequence learning using TensorFlow
  ☆1,485Updated 2 years ago
• facebookresearch / XLM
  PyTorch original implementation of Cross-lingual Language Model Pretraining.
  ☆2,921Updated 2 years ago
• facebookresearch / LASER
  Language-Agnostic SEntence Representations
  ☆3,658Updated last year
• clab / fast_align
  Simple, fast unsupervised word aligner
  ☆760Updated 3 years ago
• marian-nmt / marian
  Fast Neural Machine Translation in C++
  ☆1,398Updated 2 years ago
• microsoft / MASS
  MASS: Masked Sequence to Sequence Pre-training for Language Generation
  ☆1,120Updated 2 years ago
• EdinburghNLP / nematus
  Open-Source Neural Machine Translation in Tensorflow
  ☆803Updated 2 years ago
• mjpost / sacrebleu
  Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons
  ☆1,193Updated 2 months ago
• facebookresearch / InferSent
  InferSent sentence embeddings
  ☆2,278Updated 4 years ago
• kimiyoung / transformer-xl
  ☆3,679Updated 3 years ago
• glample / fastBPE
  Fast BPE
  ☆678Updated last year
• HIT-SCIR / ELMoForManyLangs
  Pre-trained ELMo Representations for Many Languages
  ☆1,463Updated 4 years ago
• attardi / wikiextractor
  A tool for extracting plain text from Wikipedia dumps
  ☆3,947Updated last year
• nikitakit / self-attentive-parser
  High-accuracy NLP parser with models for 11 languages.
  ☆900Updated 3 years ago
• google-research / electra
  ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
  ☆2,364Updated last year
• google-research / multilingual-t5
  ☆1,289Updated 2 years ago
• facebookresearch / MUSE
  A library for Multilingual Unsupervised or Supervised word Embeddings
  ☆3,231Updated 3 years ago
• namisan / mt-dnn
  Multi-Task Deep Neural Networks for Natural Language Understanding
  ☆2,258Updated last year
• PetrochukM / PyTorch-NLP
  Basic Utilities for PyTorch Natural Language Processing (NLP)
  ☆2,220Updated 2 years ago
• zihangdai / xlnet
  XLNet: Generalized Autoregressive Pretraining for Language Understanding
  ☆6,178Updated 2 years ago
• allenai / bilm-tf
  Tensorflow implementation of contextualized word representations from bi-directional language models
  ☆1,613Updated 2 years ago
• nyu-mll / jiant
  jiant is an nlp toolkit
  ☆1,670Updated 2 years ago
• epfml / sent2vec
  General purpose unsupervised sentence representations
  ☆1,204Updated 3 years ago
• jiesutd / NCRFpp
  NCRF++, a Neural Sequence Labeling Toolkit. Easy use to any sequence labeling tasks (e.g. NER, POS, Segmentation). It includes character …
  ☆1,896Updated 3 years ago
• OpenNMT / OpenNMT-py
  Open Source Neural Machine Translation and (Large) Language Models in PyTorch
  ☆6,972Updated last month
• parlance / ctcdecode
  PyTorch CTC Decoder bindings
  ☆852Updated last year