huggingface / lm-evaluation-harness
A framework for few-shot evaluation of language models.
☆17Updated last week
Related projects ⓘ
Alternatives and complementary repositories for lm-evaluation-harness
- Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.☆64Updated 3 weeks ago
- Codebase accompanying the Summary of a Haystack paper.☆72Updated last month
- Official implementation for 'Extending LLMs’ Context Window with 100 Samples'☆73Updated 9 months ago
- ☆73Updated 10 months ago
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆49Updated last week
- ☆37Updated last year
- ☆92Updated last month
- Official code for "MAmmoTH2: Scaling Instructions from the Web" [NeurIPS 2024]☆124Updated 2 weeks ago
- ☆43Updated 3 months ago
- Small and Efficient Mathematical Reasoning LLMs☆71Updated 9 months ago
- Retrieval Augmented Generation Generalized Evaluation Dataset☆51Updated last month
- Using open source LLMs to build synthetic datasets for direct preference optimization☆40Updated 8 months ago
- Code for NeurIPS LLM Efficiency Challenge☆53Updated 7 months ago
- [Data + code] ExpertQA : Expert-Curated Questions and Attributed Answers☆122Updated 7 months ago
- ☆44Updated 2 months ago
- Retrieval-Augmented Generation battle!☆44Updated last month
- [NeurIPS 2024] Train LLMs with diverse system messages reflecting individualized preferences to generalize to unseen system messages☆36Updated last month
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆22Updated 8 months ago
- ☆111Updated last month
- Benchmarking library for RAG☆113Updated this week
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Updated last year
- LongEmbed: Extending Embedding Models for Long Context Retrieval (EMNLP 2024)☆114Updated this week
- ☆56Updated 8 months ago
- Datasets collection and preprocessings framework for NLP extreme multitask learning☆149Updated 4 months ago
- Sakura-SOLAR-DPO: Merge, SFT, and DPO☆115Updated 10 months ago
- Scalable Meta-Evaluation of LLMs as Evaluators☆41Updated 8 months ago
- 🚢 Data Toolkit for Sailor Language Models☆81Updated 4 months ago
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆92Updated last year
- Finetune mistral-7b-instruct for sentence embeddings☆70Updated 6 months ago
- [EMNLP 2024] A Retrieval Benchmark for Scientific Literature Search☆61Updated 4 months ago