Alternatives and similar repositories for fastlangid

Users that are interested in fastlangid are comparing it to the libraries listed below

• bitextor / bifixer
  Tool to fix bitexts and tag near-duplicates for removal
  ☆31Updated 6 months ago
• AU-DIS / LSTM_langid
  Source code for the Apple reproduction
  ☆32Updated 4 years ago
• daltonfury42 / truecase
  A python true casing utility that restores case information for texts
  ☆89Updated 2 years ago
• mayhewsw / pytorch-truecaser
  A simple neural truecaser written in pytorch and allennlp.
  ☆33Updated last year
• oscar-project / goclassy
  An asynchronous concurrent pipeline for classifying Common Crawl based on fastText's pipeline.
  ☆86Updated 4 years ago
• aboSamoor / pycld2
  ☆173Updated 5 months ago
• bsolomon1124 / pycld3
  Python3 bindings for the Compact Language Detector v3 (CLD3)
  ☆154Updated 2 years ago
• lgessler / microbert
  A tiny BERT for low-resource monolingual models
  ☆31Updated 11 months ago
• TurkuNLP / wikibert
  BERT models for many languages created from Wikipedia texts
  ☆33Updated 5 years ago
• MartinoMensio / spacy-sentence-bert
  Sentence transformers models for SpaCy
  ☆107Updated 2 years ago
• snukky / wikiedits
  Automatic extraction of edited sentences from text edition histories.
  ☆83Updated 3 years ago
• mingruimingrui / fast-mosestokenizer
  c++ mosestokenizer
  ☆18Updated last year
• indix / whatthelang
  Lightning Fast Language Prediction 🚀
  ☆167Updated last week
• codogogo / towerparse
  Tower Parse: Low-Resource Dependency Parsing via Hierarchical Source Selection
  ☆15Updated 4 years ago
• mingruimingrui / ICU-tokenizer
  ICU based universal language tokenizer
  ☆33Updated 3 years ago
• bitextor / bicleaner
  Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.
  ☆158Updated last year
• kpu / preprocess
  Corpus preprocessing
  ☆98Updated last year
• Helsinki-NLP / OpusFilter
  OpusFilter - Parallel corpus processing toolkit
  ☆110Updated 3 weeks ago
• laurieburchell / open-lid-dataset
  Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)
  ☆74Updated 4 months ago
• zaemyung / sentsplit
  A flexible sentence segmentation library using CRF model and regex rules
  ☆29Updated last year
• ChenghaoMou / embeddings
  zero-vocab or low-vocab embeddings
  ☆18Updated 3 years ago
• shyyhs / CourseraParallelCorpusMining
  Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation
  ☆14Updated last year
• helboukkouri / character-bert-pretraining
  Code for pre-training CharacterBERT models (as well as BERT models).
  ☆34Updated 3 years ago
• MicrosoftTranslator / NTREX
  NTREX -- News Test References for MT Evaluation
  ☆85Updated last year
• uhh-lt / targer
  A web application tagging and retrieval of arguments in text
  ☆29Updated 2 years ago
• snakers4 / emoji-sentiment-dataset
  ☆15Updated 6 years ago
• kpu / fasterText
  Library for fast text representation and classification.
  ☆31Updated last year
• bjascob / pyInflect
  A python module for word inflections designed for use with spaCy.
  ☆93Updated 5 years ago
• nreimers / truecaser
  Language independent truecaser in Python.
  ☆160Updated 3 years ago
• jwieting / para-nmt-50m
  Pre-trained models and code and data to train and use models from "Pushing the Limits of Paraphrastic Sentence Embeddings with Millions o…
  ☆103Updated last year