Alternatives and similar repositories for web-languages:

Users that are interested in web-languages are comparing it to the libraries listed below

• cisnlp / GlotLID
  Language Identification with Support for More Than 2000 Labels -- EMNLP 2023
  ☆118Updated 3 months ago
• r-three / common-pile
  Repo to hold code and track issues for the collection of permissively licensed data
  ☆23Updated 2 months ago
• bigscience-workshop / lam
  Libraries, Archives and Museums (LAM)
  ☆82Updated 2 years ago
• malteos / llm-datasets
  A collection of datasets for language model pretraining including scripts for downloading, preprocesssing, and sampling.
  ☆56Updated 7 months ago
• UBC-NLP / afrolid
  AfroLID, a powerful neural toolkit for African languages identification which covers 517 African languages.
  ☆31Updated last year
• laurieburchell / open-lid-dataset
  Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)
  ☆70Updated 10 months ago
• MilaNLProc / simple-generation
  A python package to run inference with HuggingFace language and vision-language checkpoints wrapping many convenient features.
  ☆26Updated 5 months ago
• wjbmattingly / spacyex
  SpaCyEx allows the creation of spaCy Matcher patterns with RegEx like syntax.
  ☆59Updated 10 months ago
• davidberenstein1957 / spacy-setfit
  This repository contains an easy and intuitive approach to use SetFit in combination with spaCy.
  ☆76Updated last year
• google-research / metricx
  ☆83Updated 2 months ago
• Pleias / marginalia
  ☆67Updated 11 months ago
• MinishLab / tokenlearn
  Pre-train Static Word Embeddings
  ☆47Updated last month
• flairNLP / CleanCoNLL
  The CleanCoNLL dataset from our EMNLP 2023 paper where we corrected annotation errors and inconsistencies in CoNLL-03.
  ☆23Updated 8 months ago
• bltlab / paranames
  ParaNames: A multilingual resource for parallel names
  ☆30Updated 9 months ago
• opinionscience / BERTransfer
  A BERT-based application for reusable text classification at scale
  ☆38Updated last year
• KennethEnevoldsen / spacy-wrap
  spaCy-wrap is a wrapper library for spaCy for including fine-tuned transformers from Huggingface in your spaCy pipeline allowing you to i…
  ☆46Updated 10 months ago
• oscar-project / ungoliant
  The pipeline for the OSCAR corpus
  ☆166Updated last year
• mbanon / fastspell
  Targetted language identifier, based on FastText and Hunspell.
  ☆34Updated 2 weeks ago
• mhulden / pyfoma
  Python Finite-State Toolkit
  ☆51Updated this week
• Knowledgator / GLiClass
  Generalist and Lightweight Model for Text Classification
  ☆87Updated this week
• paulrinckens / timexy
  A spaCy custom component that extracts and normalizes temporal expressions
  ☆54Updated 2 years ago
• MeLeLBGU / SaGe
  Code for SaGe subword tokenizer (EACL 2023)
  ☆24Updated 3 months ago
• kpu / fasterText
  Library for fast text representation and classification.
  ☆28Updated last year
• webis-de / lightning-ir
  One-stop shop for running and fine-tuning transformer-based language models for retrieval
  ☆47Updated this week
• CPJKU / wechsel
  Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.
  ☆77Updated 5 months ago
• explosion / spacy-experimental
  🧪 Cutting-edge experimental spaCy components and features
  ☆96Updated 10 months ago
• cisnlp / Glot500
  Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023
  ☆99Updated 10 months ago
• hplt-project / OpusCleaner
  OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.
  ☆49Updated last month
• EuroEval / EuroEval
  The robust European language model benchmark.
  ☆81Updated this week