Alternatives and similar repositories for pile_dedupe:

Users that are interested in pile_dedupe are comparing it to the libraries listed below

• nkandpa2 / long_tail_knowledge
  Repo for the paper "Large Language Models Struggle to Learn Long-Tail Knowledge"
  ☆74Updated last year
• swj0419 / in-context-pretraining
  ☆47Updated 10 months ago
• gmftbyGMFTBY / Rep-Dropout
  [NeurIPS 2023] Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective
  ☆30Updated last year
• kernelmachine / demix
  DEMix Layers for Modular Language Modeling
  ☆53Updated 3 years ago
• wzhouad / context-faithful-llm
  Code and data for paper "Context-faithful Prompting for Large Language Models".
  ☆39Updated last year
• ndaheim / faithful-dialogue
  ☆23Updated last year
• jzbjyb / ReAtt
  Retrieval as Attention
  ☆83Updated 2 years ago
• allenai / data-efficient-finetuning
  Code for paper 'Data-Efficient FineTuning'
  ☆29Updated last year
• KwanWaiChung / M4LE
  Code for M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models
  ☆22Updated 6 months ago
• anadim / the-little-retrieval-test
  ☆34Updated last year
• yuzhaouoe / pretraining-data-packing
  [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training
  ☆19Updated 6 months ago
• frankxu2004 / knnlm-why
  Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"
  ☆56Updated 2 years ago
• OpenLMLab / LongWanjuan
  Towards Systematic Measurement for Long Text Quality
  ☆31Updated 5 months ago
• sail-sg / symbolic-instruction-tuning
  The official repository for the paper "From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuning".
  ☆63Updated last year
• swj0419 / kNN_prompt
  TBC
  ☆26Updated 2 years ago
• thunlp / ConvDR
  Code repo for SIGIR 2021 paper "Few-Shot Conversational Dense Retrieval"
  ☆41Updated 3 years ago
• salesforce / factualNLG
  Code for the arXiv paper: "LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond"
  ☆59Updated 3 weeks ago
• ChiyuSONG / dynamics-of-instruction-tuning
  ☆16Updated 11 months ago
• jzbjyb / lm-calibration
  ☆33Updated 3 years ago
• microsoft / Efficient-Large-LM-Trainer
  ☆36Updated 6 months ago
• WebQnA / WebQA
  ☆47Updated last month
• allenai / bff
  ☆38Updated 10 months ago
• ielab / PromptReps
  Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval
  ☆40Updated 3 months ago
• SimengSun / alpaca_farm_lora
  ☆22Updated last year
• ryokamoi / wice
  This repository contains the dataset and code for "WiCE: Real-World Entailment for Claims in Wikipedia" in EMNLP 2023.
  ☆40Updated last year
• castorini / dhr
  Dense hybrid representations for text retrieval
  ☆62Updated last year
• project-miracl / hagrid
  A Human-LLM Collaborative Dataset for Generative Information-seeking with Attribution
  ☆30Updated last year
• jasonwei20 / chain-of-thought-prompting
  ☆38Updated 2 years ago
• McGill-NLP / retriever-lm-reasoning
  Code for "Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model", EMNLP Findings 20…
  ☆27Updated last year
• oriram / spider
  ☆55Updated 2 years ago