Alternatives and similar repositories for infini-gram

Users that are interested in infini-gram are comparing it to the libraries listed below

• google-research-datasets / swim-ir
  SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…
  ☆48Updated last year
• castorini / hf-spacerini
  Plug-and-play Search Interfaces with Pyserini and Hugging Face
  ☆32Updated last year
• MeLeLBGU / SaGe
  Code for SaGe subword tokenizer (EACL 2023)
  ☆25Updated 7 months ago
• MinishLab / tokenlearn
  Pre-train Static Word Embeddings
  ☆84Updated last month
• Knowledgator / FlashDeBERTa
  Trully flash implementation of DeBERTa disentangled attention mechanism.
  ☆61Updated last month
• ielab / Starbucks
  Starbucks: Improved Training for 2D Matryoshka Embeddings
  ☆21Updated 2 weeks ago
• facebookresearch / lss_eval
  This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…
  ☆31Updated last year
• jxmorris12 / bm25_pt
  minimal pytorch implementation of bm25 (with sparse tensors)
  ☆102Updated last year
• illuin-tech / contextual-embeddings
  Model implementation for the contextual embeddings project
  ☆33Updated last month
• luyug / magix
  Supercharge huggingface transformers with model parallelism.
  ☆77Updated 9 months ago
• allenai / EmbeddingRecycling
  Embedding Recycling for Language models
  ☆38Updated 2 years ago
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated last year
• jfkback / hypencoder-paper
  Official Repository for "Hypencoder: Hypernetworks for Information Retrieval"
  ☆25Updated 4 months ago
• mungg / FABLES
  ☆57Updated 9 months ago
• HazyResearch / aioli
  Aioli: A unified optimization framework for language model data mixing
  ☆27Updated 5 months ago
• Aleph-Alpha-Research / trigrams
  ☆56Updated 2 months ago
• kevinwu23 / StanfordFineTuneBench
  ☆30Updated 8 months ago
• EleutherAI / semantic-memorization
  ☆44Updated 7 months ago
• allenai / bff
  ☆38Updated last year
• ZeroSumEval / ZeroSumEval
  A framework for pitting LLMs against each other in an evolving library of games ⚔
  ☆32Updated 2 months ago
• TheDuckAI / arb
  Advanced Reasoning Benchmark Dataset for LLMs
  ☆47Updated last year
• srush / LLM-Talk
  ☆51Updated last year
• trapoom555 / Language-Model-STS-CFT
  Improving Text Embedding of Language Models Using Contrastive Fine-tuning
  ☆64Updated 11 months ago
• hadasah / btm
  ☆74Updated last year
• castorini / umbrela
  ☆50Updated 4 months ago
• krypticmouse / matryoshka-representation-learning
  PyTorch implementation for MRL
  ☆18Updated last year
• allenai / infinigram-api
  ☆69Updated last month
• princeton-nlp / LitSearch
  [EMNLP 2024] A Retrieval Benchmark for Scientific Literature Search
  ☆90Updated 7 months ago
• open-lm-engine / lm-engine
  LM engine is a library for pretraining/finetuning LLMs
  ☆59Updated this week
• EleutherAI / improved-t5
  Experiments for efforts to train a new and improved t5
  ☆76Updated last year