Alternatives and similar repositories for llm-datasets:

Users that are interested in llm-datasets are comparing it to the libraries listed below

• chaitanyamalaviya / ExpertQA
  [Data + code] ExpertQA : Expert-Curated Questions and Attributed Answers
  ☆126Updated last year
• huggingface / that_is_good_data
  ☆65Updated last year
• sileod / tasksource
  Datasets collection and preprocessings framework for NLP extreme multitask learning
  ☆178Updated 3 months ago
• allenai / peS2o
  Pretraining Efficiently on S2ORC!
  ☆160Updated 5 months ago
• webis-de / lightning-ir
  One-stop shop for running and fine-tuning transformer-based language models for retrieval
  ☆52Updated last week
• malteos / clp-transfer
  Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning
  ☆30Updated 2 years ago
• LAGoM-NLP / transtokenizer
  ☆44Updated 2 months ago
• google-research-datasets / swim-ir
  SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…
  ☆48Updated last year
• mungg / FABLES
  ☆57Updated 6 months ago
• castorini / ragnarok
  Retrieval-Augmented Generation battle!
  ☆49Updated 3 months ago
• princeton-nlp / LitSearch
  [EMNLP 2024] A Retrieval Benchmark for Scientific Literature Search
  ☆79Updated 4 months ago
• cisnlp / Glot500
  Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023
  ☆100Updated 11 months ago
• huggingface / olm-training
  Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.
  ☆93Updated 2 years ago
• Knowledgator / LiqFit
  Efficient few-shot learning with cross-encoders.
  ☆50Updated last year
• cisnlp / GlotLID
  💬 Language Identification with Support for More Than 2000 Labels -- EMNLP 2023
  ☆126Updated 4 months ago
• worldbank / GISTEmbed
  GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embeddings
  ☆41Updated last year
• Knowledgator / GLiClass
  Generalist and Lightweight Model for Text Classification
  ☆115Updated this week
• ikergarcia1996 / T-Projection
  T-Projection is a method to perform high-quality Annotation Projection of Sequence Labeling datasets.
  ☆12Updated last year
• flairNLP / fabricator
  [EMNLP 2023 Demo] fabricator - annotating and generating datasets with large language models.
  ☆108Updated 10 months ago
• terrierteam / pyterrier_colbert
  ☆85Updated last week
• MinishLab / tokenlearn
  Pre-train Static Word Embeddings
  ☆52Updated last month
• MeLeLBGU / SaGe
  Code for SaGe subword tokenizer (EACL 2023)
  ☆24Updated 4 months ago
• yuxiaw / OpenFactCheck
  ☆43Updated 10 months ago
• chandar-lab / NeoBERT
  ☆44Updated last month
• katzurik / NERetrieve
  ☆28Updated last year
• JoelNiklaus / LEXTREME
  This repository provides scripts for evaluating NLP models on the LEXTREME benchmark, a set of diverse multilingual tasks in legal NLP
  ☆21Updated last year
• bigscience-workshop / data_tooling
  Tools for managing datasets for governance and training.
  ☆85Updated 2 months ago
• ielab / Starbucks
  Starbucks: Improved Training for 2D Matryoshka Embeddings
  ☆19Updated 2 months ago
• lm-pub-quiz / lm-pub-quiz
  Evaluate language models using multiple choice items
  ☆13Updated 2 months ago
• thakur-nandan / income
  INCOME: An Easy Repository for Training and Evaluation of Index Compression Methods in Dense Retrieval. Includes BPR and JPQ.
  ☆24Updated last year