huggingface / lm-evaluation-harness
A framework for few-shot evaluation of language models.
☆28Updated last week
Alternatives and similar repositories for lm-evaluation-harness:
Users that are interested in lm-evaluation-harness are comparing it to the libraries listed below
- Codebase accompanying the Summary of a Haystack paper.☆76Updated 6 months ago
- ☆119Updated 6 months ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆55Updated 7 months ago
- Implementation of "SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models"☆27Updated last month
- ☆48Updated 4 months ago
- ☆57Updated 6 months ago
- [EMNLP 2024] A Retrieval Benchmark for Scientific Literature Search☆70Updated 3 months ago
- Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…☆72Updated 7 months ago
- [Data + code] ExpertQA : Expert-Curated Questions and Attributed Answers☆126Updated last year
- 🚢 Data Toolkit for Sailor Language Models☆88Updated last month
- Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.☆73Updated 5 months ago
- Official code for "MAmmoTH2: Scaling Instructions from the Web" [NeurIPS 2024]☆137Updated 5 months ago
- Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators (Liu et al.; COLM 2024)☆44Updated 2 months ago
- ☆90Updated last week
- Manage scalable open LLM inference endpoints in Slurm clusters☆253Updated 8 months ago
- ☆65Updated 4 months ago
- ☆47Updated 7 months ago
- Minimal implementation of the Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models paper (ArXiv 20232401.01335)☆29Updated last year
- Meta-CoT: Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models☆95Updated last year
- ☆74Updated last year
- ☆112Updated 6 months ago
- ☆38Updated 11 months ago
- Code for EMNLP 2024 paper "Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoning"☆53Updated 5 months ago
- The official repo for "LLoCo: Learning Long Contexts Offline"☆116Updated 9 months ago
- LongEmbed: Extending Embedding Models for Long Context Retrieval (EMNLP 2024)☆131Updated 4 months ago
- Code for PHATGOOSE introduced in "Learning to Route Among Specialized Experts for Zero-Shot Generalization"☆82Updated last year
- ☆115Updated 7 months ago
- Using open source LLMs to build synthetic datasets for direct preference optimization☆59Updated last year
- Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate"☆131Updated last month
- ☆39Updated 7 months ago