Alternatives and similar repositories for fast-mosestokenizer

Users that are interested in fast-mosestokenizer are comparing it to the libraries listed below

• marian-nmt / sotastream
  A library for data streaming and augmentation
  ☆20Updated 2 months ago
• mayhewsw / pytorch-truecaser
  A simple neural truecaser written in pytorch and allennlp.
  ☆33Updated last year
• allenai / tpu_pretrain
  LM Pretraining with PyTorch/TPU
  ☆134Updated 5 years ago
• nikitakit / sabertooth
  Standalone pre-training recipe with JAX+Flax
  ☆32Updated 2 years ago
• bitextor / bifixer
  Tool to fix bitexts and tag near-duplicates for removal
  ☆30Updated 5 months ago
• harvardnlp / hmm-lm
  ☆41Updated 4 years ago
• harvardnlp / cascaded-generation
  Cascaded Text Generation with Markov Transformers
  ☆129Updated 2 years ago
• cambridgeltl / parameter-factorization
  Factorization of the neural parameter space for zero-shot multi-lingual and multi-task transfer
  ☆39Updated 4 years ago
• allenai / sledgehammer
  ☆47Updated 5 years ago
• ofirpress / sandwich_transformer
  This repository contains the code for running the character-level Sandwich Transformers from our ACL 2020 paper on Improving Transformer …
  ☆55Updated 4 years ago
• wietsedv / gpt2-recycle
  As good as new. How to successfully recycle English GPT-2 to make models for other languages (ACL Findings 2021)
  ☆48Updated 3 years ago
• matthewfl / openfst-wrapper
  ☆28Updated 4 years ago
• jungokasai / deep-shallow
  ☆44Updated 4 years ago
• nyu-mll / pretraining-learning-curves
  The repository for the paper "When Do You Need Billions of Words of Pretraining Data?"
  ☆21Updated 4 years ago
• zomux / lanmt
  LaNMT: Latent-variable Non-autoregressive Neural Machine Translation with Deterministic Inference
  ☆80Updated 3 years ago
• carolinlawrence / BiSon
  Code for bidirectional sequence generation (BiSon) for generating from BERT pre-trained models.
  ☆51Updated 5 years ago
• seraphlabs-ca / SentenceMIM-demo
  This repo contains code to reproduce some of the results presented in the paper "SentenceMIM: A Latent Variable Language Model"
  ☆28Updated 3 years ago
• TurkuNLP / wikibert
  BERT models for many languages created from Wikipedia texts
  ☆33Updated 5 years ago
• cindyxinyiwang / SDE
  Source code for the paper "Multilingual Neural Machine Translation with Soft Decoupled Encoding"
  ☆29Updated 4 years ago
• TimDettmers / transformer-xl
  ☆64Updated 5 years ago
• jungokasai / beam_with_patience
  ☆46Updated 3 years ago
• noe / fairseq-tensorboard
  Small utility to monitor fairseq training in tensorboard
  ☆21Updated 6 years ago
• shijie-wu / neural-transducer
  This repo contains a set of neural transducer, e.g. sequence-to-sequence model, focusing on character-level tasks.
  ☆76Updated last year
• amittai / cynical
  Cynical data selection
  ☆20Updated 4 years ago
• VProv / BPE-Dropout
  An official implementation of "BPE-Dropout: Simple and Effective Subword Regularization" algorithm.
  ☆53Updated 4 years ago
• oscar-project / goclassy
  An asynchronous concurrent pipeline for classifying Common Crawl based on fastText's pipeline.
  ☆86Updated 4 years ago
• pytorch-tpu / examples
  This repository contains example code to build models on TPUs
  ☆30Updated 2 years ago
• thevasudevgupta / transformers-adapters
  This repositary hosts my experiments for the project, I did with OffNote Labs.
  ☆10Updated 4 years ago
• yanaiela / num_fh
  numeric fused-head identification and resolution
  ☆33Updated 5 years ago
• UriSha / EmbeddinglessNMT
  The implementation of "Neural Machine Translation without Embeddings", NAACL 2021
  ☆33Updated 4 years ago