Alternatives and similar repositories for terashuf

Users that are interested in terashuf are comparing it to the libraries listed below

• glample / fastBPE
  Fast BPE
  ☆678Updated last year
• soaxelbrooke / python-bpe
  Byte Pair Encoding for Python!
  ☆232Updated 3 years ago
• microsoft / infinibatch
  Efficient, check-pointed data loading for deep learning with massive data sets.
  ☆210Updated 2 years ago
• mtreviso / linear-chain-crf
  Implementation of a linear-chain CRF in PyTorch
  ☆98Updated 4 years ago
• daltonfury42 / truecase
  A python true casing utility that restores case information for texts
  ☆88Updated 3 years ago
• facebookresearch / unlikelihood_training
  Neural Text Generation with Unlikelihood Training
  ☆310Updated 4 years ago
• kawine / usif
  Implementation of unsupervised smoothed inverse frequency (Best Paper, Repl4NLP @ ACL 2018)
  ☆79Updated 6 years ago
• microsoft / fastseq
  An efficient implementation of the popular sequence models for text generation, summarization, and translation tasks. https://arxiv.org/p…
  ☆433Updated 3 years ago
• VProv / BPE-Dropout
  An official implementation of "BPE-Dropout: Simple and Effective Subword Regularization" algorithm.
  ☆53Updated 4 years ago
• oaqa / FlexNeuART
  Flexible classic and NeurAl Retrieval Toolkit
  ☆221Updated 5 months ago
• lena-voita / good-translation-wrong-in-context
  This is a repository with the data and code for the ACL 2019 paper "When a Good Translation is Wrong in Context: ..." and the EMNLP 2019 …
  ☆98Updated 5 years ago
• awslabs / mlm-scoring
  Python library & examples for Masked Language Model Scoring (ACL 2020)
  ☆347Updated 3 years ago
• rdspring1 / PyTorch_GBW_LM
  PyTorch Language Model for 1-Billion Word (LM1B / GBW) Dataset
  ☆123Updated 6 years ago
• lilt / alignment-scripts
  Scripts to preprocess training and test data and to run fast_align and giza
  ☆107Updated 4 years ago
• kpu / preprocess
  Corpus preprocessing
  ☆99Updated last year
• thammegowda / mtdata
  A tool that locates, downloads, and extracts machine translation corpora
  ☆159Updated 3 months ago
• nyu-dl / bert-gen
  ☆324Updated 3 years ago
• urvashik / knnlm
  ☆328Updated 4 years ago
• hamed-zamani / snrm
  Standalone Neural Ranking Model (SNRM)
  ☆76Updated 6 years ago
• zomux / neuralcompressor
  Embedding Quantization (Compress Word Embeddings)
  ☆85Updated 6 years ago
• allenai / tpu_pretrain
  LM Pretraining with PyTorch/TPU
  ☆137Updated 6 years ago
• hainan-xv / zipporah
  ☆42Updated 7 years ago
• tca19 / near-lossless-binarization
  This repository contains source code to binarize any real-value word embeddings into binary vectors.
  ☆47Updated 4 years ago
• studio-ousia / bpr
  Binary Passage Retriever (BPR) - an efficient passage retriever for open-domain question answering
  ☆174Updated 4 years ago
• snukky / wikiedits
  Automatic extraction of edited sentences from text edition histories.
  ☆83Updated 3 years ago
• bojone / labse
  Language-agnostic BERT Sentence Embedding (LaBSE)
  ☆153Updated 5 years ago
• grammatical / neural-naacl2018
  Neural models and instructions on how to reproduce our results for our neural grammatical error correction systems from M. Junczys-Dowmun…
  ☆88Updated 6 years ago
• zphang / bert_on_stilts
  Fork of huggingface/pytorch-pretrained-BERT for BERT on STILTs
  ☆106Updated 3 years ago
• transducens / LASERtrain
  ☆22Updated 6 years ago
• yannvgn / laserembeddings
  LASER multilingual sentence embeddings as a pip package
  ☆225Updated 2 years ago