Alternatives and similar repositories for fasterText:

Users that are interested in fasterText are comparing it to the libraries listed below

• NathanGodey / headless-lm
  Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…
  ☆25Updated 10 months ago
• MeLeLBGU / SaGe
  Code for SaGe subword tokenizer (EACL 2023)
  ☆22Updated 2 months ago
• MinishLab / tokenlearn
  Pre-train Static Word Embeddings
  ☆47Updated 3 weeks ago
• lucy3 / whos_filtered
  ☆14Updated 4 months ago
• salesforce / simplification
  ☆21Updated 3 weeks ago
• stefan-it / modern-bert-ner
  My NER Experiments with ModernBERT
  ☆17Updated last month
• allenai / EmbeddingRecycling
  Embedding Recycling for Language models
  ☆38Updated last year
• castorini / hf-spacerini
  Plug-and-play Search Interfaces with Pyserini and Hugging Face
  ☆32Updated last year
• nbroad1881 / strideformer
  Using short models to classify long texts
  ☆21Updated last year
• gautierdag / tokenizer-bench
  Code for the paper "Getting the most out of your tokenizer for pre-training and domain adaptation"
  ☆15Updated last year
• UniversalNER / UniversalNER
  ☆26Updated 6 months ago
• ielab / Starbucks
  Starbucks: Improved Training for 2D Matryoshka Embeddings
  ☆17Updated 2 weeks ago
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated last year
• mariosasko / datasets_sql
  Execute arbitrary SQL queries on 🤗 Datasets
  ☆32Updated last year
• ltgoslo / ltg-bert
  LTG-Bert
  ☆29Updated last year
• stephantul / unitoken
  Tokenization across languages. Useful as preprocessing for subword tokenization.
  ☆22Updated 2 years ago
• UKPLab / eacl2024-lagonn
  Source code and data for Like a Good Nearest Neighbor
  ☆28Updated last month
• gsarti / t5-flax-gcp
  Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP
  ☆58Updated 2 years ago
• marian-nmt / sotastream
  A library for data streaming and augmentation
  ☆20Updated 11 months ago
• argilla-io / distilabel-spin-dibt
  Repository containing the SPIN experiments on the DIBT 10k ranked prompts
  ☆24Updated 11 months ago
• CPJKU / wechsel
  Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.
  ☆77Updated 5 months ago
• pchizhov / picky_bpe
  BPE modification that implements removing of the intermediate tokens during tokenizer training.
  ☆25Updated 2 months ago
• CarperAI / decontamination
  This repository contains code for cleaning your training data of benchmark data to help combat data snooping.
  ☆25Updated last year
• thakur-nandan / income
  INCOME: An Easy Repository for Training and Evaluation of Index Compression Methods in Dense Retrieval. Includes BPR and JPQ.
  ☆22Updated last year
• thakur-nandan / sprint
  SPRINT Toolkit helps you evaluate diverse neural sparse models easily using a single click on any IR dataset.
  ☆44Updated last year
• alvarobartt / vertex-ai-huggingface-inference-toolkit
  🤗 HuggingFace Inference Toolkit for Google Cloud Vertex AI (similar to SageMaker's Inference Toolkit, but for Vertex AI and unofficial)
  ☆17Updated 11 months ago
• shuttie / embed-benchmark
  ☆15Updated last year
• tatHi / optok
  ☆11Updated 3 years ago
• allenai / cached_path
  A file utility for accessing both local and remote files through a unified interface.
  ☆37Updated last month
• jungokasai / beam_with_patience
  ☆46Updated 2 years ago